I0628 19:58:11.716121 1 options.go:220] external host was not specified, using 199.204.45.169 I0628 19:58:11.717046 1 server.go:148] Version: v1.28.13 I0628 19:58:11.717097 1 server.go:150] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" I0628 19:58:12.039001 1 shared_informer.go:311] Waiting for caches to sync for node_authorizer I0628 19:58:12.050317 1 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook. I0628 19:58:12.051609 1 plugins.go:161] Loaded 13 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,ClusterTrustBundleAttest,CertificateSubjectRestriction,ValidatingAdmissionPolicy,ValidatingAdmissionWebhook,ResourceQuota. I0628 19:58:12.051990 1 instance.go:298] Using reconciler: lease I0628 19:58:12.239892 1 handler.go:275] Adding GroupVersion apiextensions.k8s.io v1 to ResourceManager W0628 19:58:12.239974 1 genericapiserver.go:744] Skipping API apiextensions.k8s.io/v1beta1 because it has no resources. I0628 19:58:12.522876 1 handler.go:275] Adding GroupVersion v1 to ResourceManager I0628 19:58:12.523248 1 instance.go:709] API group "internal.apiserver.k8s.io" is not enabled, skipping. I0628 19:58:13.027218 1 instance.go:709] API group "resource.k8s.io" is not enabled, skipping. I0628 19:58:13.040557 1 handler.go:275] Adding GroupVersion authentication.k8s.io v1 to ResourceManager W0628 19:58:13.040586 1 genericapiserver.go:744] Skipping API authentication.k8s.io/v1beta1 because it has no resources. W0628 19:58:13.040593 1 genericapiserver.go:744] Skipping API authentication.k8s.io/v1alpha1 because it has no resources. I0628 19:58:13.041007 1 handler.go:275] Adding GroupVersion authorization.k8s.io v1 to ResourceManager W0628 19:58:13.041021 1 genericapiserver.go:744] Skipping API authorization.k8s.io/v1beta1 because it has no resources. I0628 19:58:13.041706 1 handler.go:275] Adding GroupVersion autoscaling v2 to ResourceManager I0628 19:58:13.042336 1 handler.go:275] Adding GroupVersion autoscaling v1 to ResourceManager W0628 19:58:13.042348 1 genericapiserver.go:744] Skipping API autoscaling/v2beta1 because it has no resources. W0628 19:58:13.042352 1 genericapiserver.go:744] Skipping API autoscaling/v2beta2 because it has no resources. I0628 19:58:13.043588 1 handler.go:275] Adding GroupVersion batch v1 to ResourceManager W0628 19:58:13.043604 1 genericapiserver.go:744] Skipping API batch/v1beta1 because it has no resources. I0628 19:58:13.044325 1 handler.go:275] Adding GroupVersion certificates.k8s.io v1 to ResourceManager W0628 19:58:13.044339 1 genericapiserver.go:744] Skipping API certificates.k8s.io/v1beta1 because it has no resources. W0628 19:58:13.044343 1 genericapiserver.go:744] Skipping API certificates.k8s.io/v1alpha1 because it has no resources. I0628 19:58:13.045029 1 handler.go:275] Adding GroupVersion coordination.k8s.io v1 to ResourceManager W0628 19:58:13.045043 1 genericapiserver.go:744] Skipping API coordination.k8s.io/v1beta1 because it has no resources. W0628 19:58:13.045075 1 genericapiserver.go:744] Skipping API discovery.k8s.io/v1beta1 because it has no resources. I0628 19:58:13.045755 1 handler.go:275] Adding GroupVersion discovery.k8s.io v1 to ResourceManager I0628 19:58:13.046892 1 handler.go:275] Adding GroupVersion networking.k8s.io v1 to ResourceManager W0628 19:58:13.046906 1 genericapiserver.go:744] Skipping API networking.k8s.io/v1beta1 because it has no resources. W0628 19:58:13.046912 1 genericapiserver.go:744] Skipping API networking.k8s.io/v1alpha1 because it has no resources. I0628 19:58:13.047277 1 handler.go:275] Adding GroupVersion node.k8s.io v1 to ResourceManager W0628 19:58:13.047291 1 genericapiserver.go:744] Skipping API node.k8s.io/v1beta1 because it has no resources. W0628 19:58:13.047294 1 genericapiserver.go:744] Skipping API node.k8s.io/v1alpha1 because it has no resources. I0628 19:58:13.047824 1 handler.go:275] Adding GroupVersion policy v1 to ResourceManager W0628 19:58:13.047838 1 genericapiserver.go:744] Skipping API policy/v1beta1 because it has no resources. I0628 19:58:13.049056 1 handler.go:275] Adding GroupVersion rbac.authorization.k8s.io v1 to ResourceManager W0628 19:58:13.049070 1 genericapiserver.go:744] Skipping API rbac.authorization.k8s.io/v1beta1 because it has no resources. W0628 19:58:13.049074 1 genericapiserver.go:744] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources. I0628 19:58:13.049399 1 handler.go:275] Adding GroupVersion scheduling.k8s.io v1 to ResourceManager W0628 19:58:13.049415 1 genericapiserver.go:744] Skipping API scheduling.k8s.io/v1beta1 because it has no resources. W0628 19:58:13.049419 1 genericapiserver.go:744] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources. I0628 19:58:13.051160 1 handler.go:275] Adding GroupVersion storage.k8s.io v1 to ResourceManager W0628 19:58:13.051176 1 genericapiserver.go:744] Skipping API storage.k8s.io/v1beta1 because it has no resources. W0628 19:58:13.051181 1 genericapiserver.go:744] Skipping API storage.k8s.io/v1alpha1 because it has no resources. I0628 19:58:13.051957 1 handler.go:275] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta3 to ResourceManager I0628 19:58:13.052689 1 handler.go:275] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta2 to ResourceManager W0628 19:58:13.052703 1 genericapiserver.go:744] Skipping API flowcontrol.apiserver.k8s.io/v1beta1 because it has no resources. W0628 19:58:13.052707 1 genericapiserver.go:744] Skipping API flowcontrol.apiserver.k8s.io/v1alpha1 because it has no resources. I0628 19:58:13.055447 1 handler.go:275] Adding GroupVersion apps v1 to ResourceManager W0628 19:58:13.055463 1 genericapiserver.go:744] Skipping API apps/v1beta2 because it has no resources. W0628 19:58:13.055468 1 genericapiserver.go:744] Skipping API apps/v1beta1 because it has no resources. I0628 19:58:13.056093 1 handler.go:275] Adding GroupVersion admissionregistration.k8s.io v1 to ResourceManager W0628 19:58:13.056107 1 genericapiserver.go:744] Skipping API admissionregistration.k8s.io/v1beta1 because it has no resources. W0628 19:58:13.056111 1 genericapiserver.go:744] Skipping API admissionregistration.k8s.io/v1alpha1 because it has no resources. I0628 19:58:13.056478 1 handler.go:275] Adding GroupVersion events.k8s.io v1 to ResourceManager W0628 19:58:13.056491 1 genericapiserver.go:744] Skipping API events.k8s.io/v1beta1 because it has no resources. I0628 19:58:13.090193 1 handler.go:275] Adding GroupVersion apiregistration.k8s.io v1 to ResourceManager W0628 19:58:13.090237 1 genericapiserver.go:744] Skipping API apiregistration.k8s.io/v1beta1 because it has no resources. I0628 19:58:13.504135 1 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/etc/kubernetes/pki/front-proxy-ca.crt" I0628 19:58:13.504160 1 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt" I0628 19:58:13.504449 1 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/etc/kubernetes/pki/apiserver.crt::/etc/kubernetes/pki/apiserver.key" I0628 19:58:13.504622 1 secure_serving.go:213] Serving securely on [::]:6443 I0628 19:58:13.504691 1 tlsconfig.go:240] "Starting DynamicServingCertificateController" I0628 19:58:13.504778 1 aggregator.go:164] waiting for initial CRD sync... I0628 19:58:13.504798 1 apf_controller.go:374] Starting API Priority and Fairness config controller I0628 19:58:13.504937 1 handler_discovery.go:412] Starting ResourceDiscoveryManager I0628 19:58:13.505200 1 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller I0628 19:58:13.505224 1 shared_informer.go:311] Waiting for caches to sync for cluster_authentication_trust_controller I0628 19:58:13.505243 1 gc_controller.go:78] Starting apiserver lease garbage collector I0628 19:58:13.505285 1 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt" I0628 19:58:13.505391 1 controller.go:116] Starting legacy_token_tracking_controller I0628 19:58:13.505421 1 shared_informer.go:311] Waiting for caches to sync for configmaps I0628 19:58:13.505419 1 available_controller.go:423] Starting AvailableConditionController I0628 19:58:13.505443 1 naming_controller.go:291] Starting NamingConditionController I0628 19:58:13.505483 1 controller.go:78] Starting OpenAPI AggregationController I0628 19:58:13.505538 1 controller.go:134] Starting OpenAPI controller I0628 19:58:13.505576 1 system_namespaces_controller.go:67] Starting system namespaces controller I0628 19:58:13.505594 1 controller.go:85] Starting OpenAPI V3 controller I0628 19:58:13.505425 1 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/etc/kubernetes/pki/front-proxy-ca.crt" I0628 19:58:13.505651 1 dynamic_serving_content.go:132] "Starting controller" name="aggregator-proxy-cert::/etc/kubernetes/pki/front-proxy-client.crt::/etc/kubernetes/pki/front-proxy-client.key" I0628 19:58:13.504958 1 customresource_discovery_controller.go:289] Starting DiscoveryController I0628 19:58:13.505729 1 gc_controller.go:78] Starting apiserver lease garbage collector I0628 19:58:13.505777 1 establishing_controller.go:76] Starting EstablishingController I0628 19:58:13.505825 1 nonstructuralschema_controller.go:192] Starting NonStructuralSchemaConditionController I0628 19:58:13.505870 1 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController I0628 19:58:13.505922 1 crd_finalizer.go:266] Starting CRDFinalizer I0628 19:58:13.505947 1 apiservice_controller.go:97] Starting APIServiceRegistrationController I0628 19:58:13.505974 1 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller I0628 19:58:13.506014 1 crdregistration_controller.go:111] Starting crd-autoregister controller I0628 19:58:13.506029 1 shared_informer.go:311] Waiting for caches to sync for crd-autoregister I0628 19:58:13.505456 1 cache.go:32] Waiting for caches to sync for AvailableConditionController controller I0628 19:58:13.505539 1 controller.go:80] Starting OpenAPI V3 AggregationController I0628 19:58:13.539313 1 shared_informer.go:318] Caches are synced for node_authorizer I0628 19:58:13.605226 1 apf_controller.go:379] Running API Priority and Fairness config worker I0628 19:58:13.605266 1 apf_controller.go:382] Running API Priority and Fairness periodic rebalancing process I0628 19:58:13.605417 1 shared_informer.go:318] Caches are synced for cluster_authentication_trust_controller I0628 19:58:13.605526 1 shared_informer.go:318] Caches are synced for configmaps I0628 19:58:13.606044 1 cache.go:39] Caches are synced for APIServiceRegistrationController controller I0628 19:58:13.606133 1 shared_informer.go:318] Caches are synced for crd-autoregister I0628 19:58:13.606177 1 aggregator.go:166] initial CRD sync complete... I0628 19:58:13.606215 1 autoregister_controller.go:141] Starting autoregister controller I0628 19:58:13.606239 1 cache.go:32] Waiting for caches to sync for autoregister controller I0628 19:58:13.606258 1 cache.go:39] Caches are synced for autoregister controller I0628 19:58:13.606557 1 cache.go:39] Caches are synced for AvailableConditionController controller I0628 19:58:13.607511 1 controller.go:624] quota admission added evaluator for: namespaces I0628 19:58:13.621042 1 controller.go:624] quota admission added evaluator for: leases.coordination.k8s.io I0628 19:58:14.517850 1 storage_scheduling.go:95] created PriorityClass system-node-critical with value 2000001000 I0628 19:58:14.534077 1 storage_scheduling.go:95] created PriorityClass system-cluster-critical with value 2000000000 I0628 19:58:14.534099 1 storage_scheduling.go:111] all system priority classes are created successfully or already exist. I0628 19:58:15.171017 1 controller.go:624] quota admission added evaluator for: roles.rbac.authorization.k8s.io I0628 19:58:15.225208 1 controller.go:624] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io I0628 19:58:15.324155 1 alloc.go:330] "allocated clusterIPs" service="default/kubernetes" clusterIPs={"IPv4":"10.96.0.1"} W0628 19:58:15.334526 1 lease.go:263] Resetting endpoints for master service "kubernetes" to [199.204.45.169] I0628 19:58:15.335572 1 controller.go:624] quota admission added evaluator for: endpoints I0628 19:58:15.341935 1 controller.go:624] quota admission added evaluator for: endpointslices.discovery.k8s.io I0628 19:58:16.208434 1 controller.go:624] quota admission added evaluator for: serviceaccounts I0628 19:58:17.123565 1 controller.go:624] quota admission added evaluator for: deployments.apps I0628 19:58:17.138738 1 alloc.go:330] "allocated clusterIPs" service="kube-system/kube-dns" clusterIPs={"IPv4":"10.96.0.10"} I0628 19:58:17.150637 1 controller.go:624] quota admission added evaluator for: daemonsets.apps I0628 19:58:21.184512 1 controller.go:624] quota admission added evaluator for: controllerrevisions.apps I0628 19:58:21.404950 1 controller.go:624] quota admission added evaluator for: replicasets.apps I0628 19:58:32.126670 1 trace.go:236] Trace[1129066487]: "Update" accept:application/json, */*,audit-id:736b8a4a-bba8-4971-861b-287222ebe806,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Jun-2026 19:58:29.970) (total time: 2155ms): Trace[1129066487]: ["GuaranteedUpdate etcd3" audit-id:736b8a4a-bba8-4971-861b-287222ebe806,key:/leases/kube-system/plndr-cp-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 2155ms (19:58:29.971) Trace[1129066487]: ---"Txn call completed" 2153ms (19:58:32.126)] Trace[1129066487]: [2.155705536s] [2.155705536s] END I0628 19:58:32.456951 1 trace.go:236] Trace[1193340118]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:c2808a3e-6740-4842-97de-e3ef5e9ccbe8,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 19:58:30.158) (total time: 2298ms): Trace[1193340118]: ---"About to write a response" 2298ms (19:58:32.456) Trace[1193340118]: [2.298579328s] [2.298579328s] END I0628 19:58:32.457339 1 trace.go:236] Trace[1405014187]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:83661dd1-de75-49b4-92d5-7ad5126389e9,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/kube-system/pods/kube-controller-manager-instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 19:58:30.392) (total time: 2064ms): Trace[1405014187]: ---"About to write a response" 2063ms (19:58:32.456) Trace[1405014187]: [2.064330196s] [2.064330196s] END I0628 19:58:32.457687 1 trace.go:236] Trace[1741044965]: "Get" accept:application/json, */*,audit-id:e6a9b22e-036d-49a8-b742-b4201251a014,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 19:58:30.190) (total time: 2267ms): Trace[1741044965]: ---"About to write a response" 2266ms (19:58:32.457) Trace[1741044965]: [2.267089586s] [2.267089586s] END I0628 19:58:32.457811 1 trace.go:236] Trace[398828059]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:331f9218-d8a9-42d7-8c58-27699fa3ea67,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 19:58:30.276) (total time: 2180ms): Trace[398828059]: ---"About to write a response" 2180ms (19:58:32.457) Trace[398828059]: [2.180954138s] [2.180954138s] END I0628 19:58:54.611001 1 trace.go:236] Trace[607933659]: "Create" accept:application/json,audit-id:e5962adb-ffe2-4448-8ed4-e58110daaf5c,client:199.204.45.169,protocol:HTTP/2.0,resource:customresourcedefinitions,scope:resource,url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions,user-agent:Helm/3.11.2,verb:POST (28-Jun-2026 19:58:54.062) (total time: 548ms): Trace[607933659]: ["Create etcd3" audit-id:e5962adb-ffe2-4448-8ed4-e58110daaf5c,key:/apiextensions.k8s.io/customresourcedefinitions/referencegrants.gateway.networking.k8s.io,type:*apiextensions.CustomResourceDefinition,resource:customresourcedefinitions.apiextensions.k8s.io 543ms (19:58:54.067) Trace[607933659]: ---"Txn call succeeded" 540ms (19:58:54.608)] Trace[607933659]: [548.712056ms] [548.712056ms] END I0628 19:58:54.618076 1 trace.go:236] Trace[109136377]: "Create" accept:application/json,audit-id:da8cb9dd-e80a-46af-8c8d-119827294a54,client:199.204.45.169,protocol:HTTP/2.0,resource:customresourcedefinitions,scope:resource,url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions,user-agent:Helm/3.11.2,verb:POST (28-Jun-2026 19:58:54.064) (total time: 553ms): Trace[109136377]: ["Create etcd3" audit-id:da8cb9dd-e80a-46af-8c8d-119827294a54,key:/apiextensions.k8s.io/customresourcedefinitions/xmeshes.gateway.networking.x-k8s.io,type:*apiextensions.CustomResourceDefinition,resource:customresourcedefinitions.apiextensions.k8s.io 543ms (19:58:54.074) Trace[109136377]: ---"Txn call succeeded" 542ms (19:58:54.616)] Trace[109136377]: [553.532699ms] [553.532699ms] END I0628 19:58:54.619792 1 trace.go:236] Trace[1963130258]: "Create" accept:application/json,audit-id:4dbcdfb9-e651-47d5-9239-99d4999bf09a,client:199.204.45.169,protocol:HTTP/2.0,resource:customresourcedefinitions,scope:resource,url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions,user-agent:Helm/3.11.2,verb:POST (28-Jun-2026 19:58:54.066) (total time: 552ms): Trace[1963130258]: ["Create etcd3" audit-id:4dbcdfb9-e651-47d5-9239-99d4999bf09a,key:/apiextensions.k8s.io/customresourcedefinitions/gatewayclasses.gateway.networking.k8s.io,type:*apiextensions.CustomResourceDefinition,resource:customresourcedefinitions.apiextensions.k8s.io 545ms (19:58:54.074) Trace[1963130258]: ---"Txn call succeeded" 543ms (19:58:54.617)] Trace[1963130258]: [552.903577ms] [552.903577ms] END I0628 19:58:54.620687 1 trace.go:236] Trace[2117465813]: "Create" accept:application/json,audit-id:9fa98d0d-5f74-4ccb-9336-666e689f85d7,client:199.204.45.169,protocol:HTTP/2.0,resource:customresourcedefinitions,scope:resource,url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions,user-agent:Helm/3.11.2,verb:POST (28-Jun-2026 19:58:54.065) (total time: 554ms): Trace[2117465813]: ["Create etcd3" audit-id:9fa98d0d-5f74-4ccb-9336-666e689f85d7,key:/apiextensions.k8s.io/customresourcedefinitions/tcproutes.gateway.networking.k8s.io,type:*apiextensions.CustomResourceDefinition,resource:customresourcedefinitions.apiextensions.k8s.io 530ms (19:58:54.090) Trace[2117465813]: ---"Txn call succeeded" 524ms (19:58:54.618)] Trace[2117465813]: [554.996988ms] [554.996988ms] END I0628 19:58:54.620862 1 trace.go:236] Trace[1784590276]: "Create" accept:application/json,audit-id:f2f829d3-9fb6-43f5-8efb-f74aeb82b170,client:199.204.45.169,protocol:HTTP/2.0,resource:customresourcedefinitions,scope:resource,url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions,user-agent:Helm/3.11.2,verb:POST (28-Jun-2026 19:58:54.068) (total time: 552ms): Trace[1784590276]: ["Create etcd3" audit-id:f2f829d3-9fb6-43f5-8efb-f74aeb82b170,key:/apiextensions.k8s.io/customresourcedefinitions/backendtlspolicies.gateway.networking.k8s.io,type:*apiextensions.CustomResourceDefinition,resource:customresourcedefinitions.apiextensions.k8s.io 536ms (19:58:54.084) Trace[1784590276]: ---"Txn call succeeded" 533ms (19:58:54.618)] Trace[1784590276]: [552.145343ms] [552.145343ms] END I0628 19:58:54.622842 1 trace.go:236] Trace[1254750292]: "Create" accept:application/json,audit-id:3c8d0141-235b-44db-900c-1cffcfd78164,client:199.204.45.169,protocol:HTTP/2.0,resource:customresourcedefinitions,scope:resource,url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions,user-agent:Helm/3.11.2,verb:POST (28-Jun-2026 19:58:54.066) (total time: 556ms): Trace[1254750292]: ["Create etcd3" audit-id:3c8d0141-235b-44db-900c-1cffcfd78164,key:/apiextensions.k8s.io/customresourcedefinitions/tlsroutes.gateway.networking.k8s.io,type:*apiextensions.CustomResourceDefinition,resource:customresourcedefinitions.apiextensions.k8s.io 513ms (19:58:54.108) Trace[1254750292]: ---"Txn call succeeded" 506ms (19:58:54.617)] Trace[1254750292]: [556.001316ms] [556.001316ms] END I0628 19:58:54.623424 1 trace.go:236] Trace[1290348990]: "Create" accept:application/json,audit-id:9eefdc8a-e332-4011-98f8-cea88ebf28cc,client:199.204.45.169,protocol:HTTP/2.0,resource:customresourcedefinitions,scope:resource,url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions,user-agent:Helm/3.11.2,verb:POST (28-Jun-2026 19:58:54.066) (total time: 556ms): Trace[1290348990]: ["Create etcd3" audit-id:9eefdc8a-e332-4011-98f8-cea88ebf28cc,key:/apiextensions.k8s.io/customresourcedefinitions/xbackendtrafficpolicies.gateway.networking.x-k8s.io,type:*apiextensions.CustomResourceDefinition,resource:customresourcedefinitions.apiextensions.k8s.io 539ms (19:58:54.084) Trace[1290348990]: ---"Txn call succeeded" 533ms (19:58:54.618)] Trace[1290348990]: [556.829372ms] [556.829372ms] END I0628 19:58:54.623757 1 trace.go:236] Trace[1590289349]: "Create" accept:application/json,audit-id:fa0869c4-ca24-4f55-a90b-0bac5a6d0b7b,client:199.204.45.169,protocol:HTTP/2.0,resource:customresourcedefinitions,scope:resource,url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions,user-agent:Helm/3.11.2,verb:POST (28-Jun-2026 19:58:54.066) (total time: 557ms): Trace[1590289349]: ["Create etcd3" audit-id:fa0869c4-ca24-4f55-a90b-0bac5a6d0b7b,key:/apiextensions.k8s.io/customresourcedefinitions/xlistenersets.gateway.networking.x-k8s.io,type:*apiextensions.CustomResourceDefinition,resource:customresourcedefinitions.apiextensions.k8s.io 539ms (19:58:54.084) Trace[1590289349]: ---"Txn call succeeded" 535ms (19:58:54.619)] Trace[1590289349]: [557.466924ms] [557.466924ms] END I0628 19:58:54.624550 1 trace.go:236] Trace[312358114]: "Create" accept:application/json,audit-id:81b246f6-7821-47f2-8521-c025a6dc76df,client:199.204.45.169,protocol:HTTP/2.0,resource:customresourcedefinitions,scope:resource,url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions,user-agent:Helm/3.11.2,verb:POST (28-Jun-2026 19:58:54.068) (total time: 556ms): Trace[312358114]: ["Create etcd3" audit-id:81b246f6-7821-47f2-8521-c025a6dc76df,key:/apiextensions.k8s.io/customresourcedefinitions/udproutes.gateway.networking.k8s.io,type:*apiextensions.CustomResourceDefinition,resource:customresourcedefinitions.apiextensions.k8s.io 529ms (19:58:54.094) Trace[312358114]: ---"Txn call succeeded" 521ms (19:58:54.617)] Trace[312358114]: [556.431075ms] [556.431075ms] END I0628 19:58:54.625796 1 trace.go:236] Trace[1538690173]: "Create" accept:application/json,audit-id:2fa99861-8529-42ee-9323-027296e86870,client:199.204.45.169,protocol:HTTP/2.0,resource:customresourcedefinitions,scope:resource,url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions,user-agent:Helm/3.11.2,verb:POST (28-Jun-2026 19:58:54.069) (total time: 556ms): Trace[1538690173]: [556.296562ms] [556.296562ms] END I0628 19:58:54.635648 1 trace.go:236] Trace[883006797]: "Create" accept:application/json,audit-id:e9b1f956-8860-4f0e-8c1d-50907ae63359,client:199.204.45.169,protocol:HTTP/2.0,resource:customresourcedefinitions,scope:resource,url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions,user-agent:Helm/3.11.2,verb:POST (28-Jun-2026 19:58:54.067) (total time: 567ms): Trace[883006797]: [567.759071ms] [567.759071ms] END I0628 19:58:54.658195 1 trace.go:236] Trace[1302685591]: "Create" accept:application/json,audit-id:eb106345-b36b-469c-9316-b9b5296d6d71,client:199.204.45.169,protocol:HTTP/2.0,resource:customresourcedefinitions,scope:resource,url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions,user-agent:Helm/3.11.2,verb:POST (28-Jun-2026 19:58:54.072) (total time: 585ms): Trace[1302685591]: ---"Conversion done" 34ms (19:58:54.108) Trace[1302685591]: ---"Writing http response done" 23ms (19:58:54.657) Trace[1302685591]: [585.313228ms] [585.313228ms] END I0628 19:58:54.705230 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0628 19:58:54.719834 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0628 19:58:54.741159 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0628 19:58:54.741212 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha3 to ResourceManager I0628 19:58:54.774164 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0628 19:58:54.791769 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0628 19:58:54.811157 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0628 19:58:54.836617 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0628 19:58:54.836688 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha3 to ResourceManager I0628 19:58:54.909589 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0628 19:58:54.909649 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0628 19:58:54.939138 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0628 19:58:55.094447 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0628 19:58:55.094523 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0628 19:58:55.101606 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0628 19:58:55.117971 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0628 19:58:55.118032 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0628 19:58:55.130222 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0628 19:58:55.189597 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0628 19:58:55.220266 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0628 19:58:55.256094 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0628 19:58:55.268018 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0628 19:58:55.905576 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0628 19:58:55.919580 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0628 19:58:55.990662 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0628 19:59:01.648015 1 trace.go:236] Trace[1528642777]: "Get" accept:application/json, */*,audit-id:bf07932c-89e1-4057-8e90-c9ceef1fef47,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 19:59:00.879) (total time: 768ms): Trace[1528642777]: ---"About to write a response" 767ms (19:59:01.647) Trace[1528642777]: [768.161873ms] [768.161873ms] END I0628 19:59:01.648231 1 trace.go:236] Trace[352295175]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:0e4722aa-d549-4ad5-8e79-c607c881b55c,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 19:59:00.894) (total time: 753ms): Trace[352295175]: ---"About to write a response" 753ms (19:59:01.648) Trace[352295175]: [753.693845ms] [753.693845ms] END I0628 19:59:01.648821 1 trace.go:236] Trace[1511457228]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:55a1667f-db81-4f7c-bba7-dd04d85bbd0c,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 19:59:00.897) (total time: 751ms): Trace[1511457228]: ---"About to write a response" 751ms (19:59:01.648) Trace[1511457228]: [751.273209ms] [751.273209ms] END I0628 19:59:01.678700 1 trace.go:236] Trace[128380223]: "Create" accept:application/json,audit-id:8f7941f5-74c3-4c0e-a51f-309df61ce0a6,client:199.204.45.169,protocol:HTTP/2.0,resource:mutatingwebhookconfigurations,scope:resource,url:/apis/admissionregistration.k8s.io/v1/mutatingwebhookconfigurations,user-agent:Helm/3.11.2,verb:POST (28-Jun-2026 19:59:00.953) (total time: 724ms): Trace[128380223]: ["Create etcd3" audit-id:8f7941f5-74c3-4c0e-a51f-309df61ce0a6,key:/mutatingwebhookconfigurations/envoy-gateway-topology-injector.envoy-gateway-system,type:*admissionregistration.MutatingWebhookConfiguration,resource:mutatingwebhookconfigurations.admissionregistration.k8s.io 722ms (19:59:00.956) Trace[128380223]: ---"Txn call succeeded" 721ms (19:59:01.678)] Trace[128380223]: [724.722222ms] [724.722222ms] END I0628 19:59:01.678827 1 trace.go:236] Trace[2080469649]: "Update" accept:application/json, */*,audit-id:8ec645a9-8c3e-4bc1-9c3e-d5e8afb2104d,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Jun-2026 19:59:00.908) (total time: 770ms): Trace[2080469649]: ["GuaranteedUpdate etcd3" audit-id:8ec645a9-8c3e-4bc1-9c3e-d5e8afb2104d,key:/leases/kube-system/plndr-cp-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 770ms (19:59:00.908) Trace[2080469649]: ---"Txn call completed" 769ms (19:59:01.678)] Trace[2080469649]: [770.392245ms] [770.392245ms] END I0628 19:59:02.698864 1 trace.go:236] Trace[240956003]: "Delete" accept:application/json,audit-id:30c2c7dc-48f8-443d-ae0b-c484bf750f50,client:199.204.45.169,protocol:HTTP/2.0,resource:jobs,scope:resource,url:/apis/batch/v1/namespaces/envoy-gateway-system/jobs/envoy-gateway-gateway-helm-certgen,user-agent:Helm/3.11.2,verb:DELETE (28-Jun-2026 19:59:01.689) (total time: 1009ms): Trace[240956003]: [1.009209403s] [1.009209403s] END I0628 19:59:02.698985 1 trace.go:236] Trace[1997465245]: "Update" accept:application/json, */*,audit-id:cd1e65a1-6e95-413b-882d-2514aab905d2,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Jun-2026 19:59:01.650) (total time: 1048ms): Trace[1997465245]: ["GuaranteedUpdate etcd3" audit-id:cd1e65a1-6e95-413b-882d-2514aab905d2,key:/leases/kube-system/plndr-svcs-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 1047ms (19:59:01.651) Trace[1997465245]: ---"Txn call completed" 1046ms (19:59:02.698)] Trace[1997465245]: [1.048012305s] [1.048012305s] END I0628 19:59:02.698864 1 trace.go:236] Trace[323154386]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:3a85e563-063d-43ca-b31c-72f33a07f6c0,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (28-Jun-2026 19:59:01.650) (total time: 1047ms): Trace[323154386]: ["GuaranteedUpdate etcd3" audit-id:3a85e563-063d-43ca-b31c-72f33a07f6c0,key:/leases/kube-system/kube-scheduler,type:*coordination.Lease,resource:leases.coordination.k8s.io 1047ms (19:59:01.651) Trace[323154386]: ---"Txn call completed" 1046ms (19:59:02.698)] Trace[323154386]: [1.047847943s] [1.047847943s] END I0628 19:59:02.698864 1 trace.go:236] Trace[195519350]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:1ddc9294-dcc7-41c5-b570-4499f2eb0e3f,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (28-Jun-2026 19:59:01.650) (total time: 1048ms): Trace[195519350]: ["GuaranteedUpdate etcd3" audit-id:1ddc9294-dcc7-41c5-b570-4499f2eb0e3f,key:/leases/kube-system/kube-controller-manager,type:*coordination.Lease,resource:leases.coordination.k8s.io 1048ms (19:59:01.650) Trace[195519350]: ---"Txn call completed" 1046ms (19:59:02.698)] Trace[195519350]: [1.048319452s] [1.048319452s] END I0628 19:59:05.035768 1 trace.go:236] Trace[169914781]: "Get" accept:application/json, */*,audit-id:1afafd0d-86fb-47c3-b50c-dc57a349f84f,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 19:59:03.681) (total time: 1354ms): Trace[169914781]: ---"About to write a response" 1353ms (19:59:05.034) Trace[169914781]: [1.354293676s] [1.354293676s] END I0628 19:59:05.070599 1 trace.go:236] Trace[237400358]: "Update" accept:application/json, */*,audit-id:c70bfae2-d6bf-4fff-b6b4-73c8c183af25,client:199.204.45.169,protocol:HTTP/2.0,resource:secrets,scope:resource,url:/api/v1/namespaces/envoy-gateway-system/secrets/sh.helm.release.v1.envoy-gateway.v1,user-agent:Helm/3.11.2,verb:PUT (28-Jun-2026 19:59:03.051) (total time: 2019ms): Trace[237400358]: ---"Conversion done" 20ms (19:59:03.077) Trace[237400358]: ["GuaranteedUpdate etcd3" audit-id:c70bfae2-d6bf-4fff-b6b4-73c8c183af25,key:/secrets/envoy-gateway-system/sh.helm.release.v1.envoy-gateway.v1,type:*core.Secret,resource:secrets 1992ms (19:59:03.077) Trace[237400358]: ---"Txn call completed" 1949ms (19:59:05.034)] Trace[237400358]: ---"Writing http response done" 35ms (19:59:05.070) Trace[237400358]: [2.019226864s] [2.019226864s] END I0628 19:59:05.091334 1 controller.go:624] quota admission added evaluator for: jobs.batch I0628 19:59:06.270461 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0628 19:59:06.305265 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0628 19:59:06.467259 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0628 19:59:06.943245 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0628 19:59:06.955995 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0628 19:59:07.063715 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0628 19:59:07.469351 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0628 19:59:07.688888 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0628 19:59:08.893393 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0628 19:59:09.896249 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0628 19:59:10.278199 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager W0628 19:59:11.200400 1 dispatcher.go:210] Failed calling webhook, failing open topology.webhook.gateway.envoyproxy.io: failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": service "envoy-gateway" not found E0628 19:59:11.200444 1 dispatcher.go:214] failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": service "envoy-gateway" not found I0628 19:59:14.250992 1 controller.go:624] quota admission added evaluator for: ciliumendpoints.cilium.io I0628 19:59:20.383936 1 trace.go:236] Trace[1702705671]: "Update" accept:application/json, */*,audit-id:55068bb7-50b4-48df-9731-43d61b35bb3f,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Jun-2026 19:59:19.160) (total time: 1223ms): Trace[1702705671]: ["GuaranteedUpdate etcd3" audit-id:55068bb7-50b4-48df-9731-43d61b35bb3f,key:/leases/kube-system/plndr-svcs-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 1223ms (19:59:19.160) Trace[1702705671]: ---"Txn call completed" 1221ms (19:59:20.383)] Trace[1702705671]: [1.22342551s] [1.22342551s] END I0628 19:59:20.385232 1 trace.go:236] Trace[1827092175]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:77d6c647-8224-4cc1-839f-33ad533fb09e,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (28-Jun-2026 19:59:19.160) (total time: 1224ms): Trace[1827092175]: ["GuaranteedUpdate etcd3" audit-id:77d6c647-8224-4cc1-839f-33ad533fb09e,key:/leases/kube-system/kube-scheduler,type:*coordination.Lease,resource:leases.coordination.k8s.io 1224ms (19:59:19.161) Trace[1827092175]: ---"Txn call completed" 1223ms (19:59:20.385)] Trace[1827092175]: [1.224225424s] [1.224225424s] END I0628 19:59:20.385307 1 trace.go:236] Trace[1032958392]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:6093815e-108b-4413-a76c-ecee54a98dc1,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (28-Jun-2026 19:59:19.160) (total time: 1224ms): Trace[1032958392]: ["GuaranteedUpdate etcd3" audit-id:6093815e-108b-4413-a76c-ecee54a98dc1,key:/leases/kube-system/kube-controller-manager,type:*coordination.Lease,resource:leases.coordination.k8s.io 1224ms (19:59:19.161) Trace[1032958392]: ---"Txn call completed" 1223ms (19:59:20.385)] Trace[1032958392]: [1.224276426s] [1.224276426s] END I0628 19:59:20.386404 1 trace.go:236] Trace[587171278]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:26df169a-95be-46c3-967e-45d1c234f302,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/envoy-gateway-system/pods/envoy-gateway-gateway-helm-certgen-f66zb,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 19:59:19.462) (total time: 923ms): Trace[587171278]: ---"About to write a response" 923ms (19:59:20.386) Trace[587171278]: [923.503474ms] [923.503474ms] END I0628 19:59:20.387542 1 trace.go:236] Trace[105463219]: "Get" accept:application/json, */*,audit-id:0a7a71ef-1f19-4616-a61c-74105fb2f872,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 19:59:19.163) (total time: 1223ms): Trace[105463219]: ---"About to write a response" 1223ms (19:59:20.387) Trace[105463219]: [1.22397006s] [1.22397006s] END I0628 19:59:23.573630 1 alloc.go:330] "allocated clusterIPs" service="envoy-gateway-system/envoy-gateway" clusterIPs={"IPv4":"10.100.65.122"} W0628 19:59:23.610097 1 dispatcher.go:210] Failed calling webhook, failing open topology.webhook.gateway.envoyproxy.io: failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": dial tcp 10.100.65.122:9443: connect: connection refused E0628 19:59:23.610150 1 dispatcher.go:214] failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": dial tcp 10.100.65.122:9443: connect: connection refused I0628 19:59:36.869789 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0628 19:59:36.884342 1 handler.go:275] Adding GroupVersion acme.cert-manager.io v1 to ResourceManager I0628 19:59:36.900995 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0628 19:59:36.916476 1 alloc.go:330] "allocated clusterIPs" service="cert-manager/cert-manager-webhook" clusterIPs={"IPv4":"10.102.151.133"} I0628 19:59:36.930738 1 alloc.go:330] "allocated clusterIPs" service="cert-manager/cert-manager" clusterIPs={"IPv4":"10.104.37.249"} I0628 19:59:36.937572 1 handler.go:275] Adding GroupVersion acme.cert-manager.io v1 to ResourceManager I0628 19:59:36.992485 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0628 19:59:37.039828 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0628 19:59:48.785524 1 controller.go:624] quota admission added evaluator for: certificates.cert-manager.io I0628 19:59:56.238434 1 controller.go:624] quota admission added evaluator for: certificaterequests.cert-manager.io I0628 20:00:14.388837 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller-metrics" clusterIPs={"IPv4":"10.100.228.66"} I0628 20:00:14.440599 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller" clusterIPs={"IPv4":"10.96.12.51"} I0628 20:00:14.454784 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-defaultbackend" clusterIPs={"IPv4":"10.100.218.62"} I0628 20:00:14.462712 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller-admission" clusterIPs={"IPv4":"10.101.5.183"} I0628 20:00:23.805554 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0628 20:00:23.823947 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0628 20:00:23.847568 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0628 20:00:23.867406 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0628 20:00:23.889211 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0628 20:00:23.908317 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0628 20:00:23.935130 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0628 20:00:23.956379 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0628 20:00:23.986063 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0628 20:00:24.007723 1 handler.go:275] Adding GroupVersion rabbitmq.com v1alpha1 to ResourceManager I0628 20:00:24.031220 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0628 20:00:24.056709 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0628 20:00:24.076333 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0628 20:00:25.455673 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0628 20:00:29.343689 1 controller.go:624] quota admission added evaluator for: networkpolicies.networking.k8s.io I0628 20:00:29.557276 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-messaging-topology-operator-webhook" clusterIPs={"IPv4":"10.101.234.211"} I0628 20:00:29.613463 1 controller.go:624] quota admission added evaluator for: issuers.cert-manager.io I0628 20:00:32.127680 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0628 20:00:32.158597 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0628 20:00:32.788224 1 handler.go:275] Adding GroupVersion pxc.percona.com v1-10-0 to ResourceManager I0628 20:00:32.788322 1 handler.go:275] Adding GroupVersion pxc.percona.com v1-11-0 to ResourceManager I0628 20:00:32.788401 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0628 20:00:40.353676 1 controller.go:624] quota admission added evaluator for: perconaxtradbclusters.pxc.percona.com I0628 20:00:52.438445 1 trace.go:236] Trace[1406285970]: "List(recursive=true) etcd3" audit-id:,key:/pxc.percona.com/perconaxtradbclusterrestores,resourceVersion:,resourceVersionMatch:,limit:10000,continue: (28-Jun-2026 20:00:51.936) (total time: 502ms): Trace[1406285970]: [502.15068ms] [502.15068ms] END I0628 20:00:52.438518 1 trace.go:236] Trace[176754018]: "List" accept:application/vnd.kubernetes.protobuf;as=PartialObjectMetadataList;g=meta.k8s.io;v=v1,application/json;as=PartialObjectMetadataList;g=meta.k8s.io;v=v1,application/json,audit-id:2faa02b2-6248-4694-8aef-b58b8aadb840,client:199.204.45.169,protocol:HTTP/2.0,resource:perconaxtradbclusterrestores,scope:cluster,url:/apis/pxc.percona.com/v1/perconaxtradbclusterrestores,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/metadata-informers,verb:LIST (28-Jun-2026 20:00:51.936) (total time: 501ms): Trace[176754018]: ["List(recursive=true) etcd3" audit-id:2faa02b2-6248-4694-8aef-b58b8aadb840,key:/pxc.percona.com/perconaxtradbclusterrestores,resourceVersion:0,resourceVersionMatch:,limit:500,continue: 501ms (20:00:51.936)] Trace[176754018]: [501.643621ms] [501.643621ms] END I0628 20:00:52.438626 1 trace.go:236] Trace[1162886546]: "Get" accept:application/json, */*,audit-id:44c263fe-2fcc-4b41-893e-a50f439c0837,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:00:51.575) (total time: 863ms): Trace[1162886546]: ---"About to write a response" 863ms (20:00:52.438) Trace[1162886546]: [863.238207ms] [863.238207ms] END I0628 20:00:52.438952 1 trace.go:236] Trace[1214413100]: "Get" accept:application/json, */*,audit-id:161288d9-11fb-4059-9382-ffc5f585fa47,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:00:51.575) (total time: 863ms): Trace[1214413100]: ---"About to write a response" 863ms (20:00:52.438) Trace[1214413100]: [863.446223ms] [863.446223ms] END I0628 20:00:52.439321 1 trace.go:236] Trace[1265151306]: "Get" accept:application/json, */*,audit-id:ef9919ea-07f4-4fa6-b48b-9db98fedcab5,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:00:51.611) (total time: 826ms): Trace[1265151306]: ---"About to write a response" 826ms (20:00:52.438) Trace[1265151306]: [826.906319ms] [826.906319ms] END I0628 20:00:52.439800 1 trace.go:236] Trace[28213727]: "List(recursive=true) etcd3" audit-id:,key:/pxc.percona.com/perconaxtradbclusterbackups,resourceVersion:,resourceVersionMatch:,limit:10000,continue: (28-Jun-2026 20:00:51.918) (total time: 520ms): Trace[28213727]: [520.975393ms] [520.975393ms] END I0628 20:00:52.439963 1 trace.go:236] Trace[1614859733]: "List" accept:application/vnd.kubernetes.protobuf, */*,audit-id:f802b234-60ca-4a65-bf73-21a0bd69d4d7,client:10.0.0.187,protocol:HTTP/2.0,resource:events,scope:namespace,url:/apis/events.k8s.io/v1/namespaces/openstack/events,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format,verb:LIST (28-Jun-2026 20:00:51.903) (total time: 536ms): Trace[1614859733]: ["List(recursive=true) etcd3" audit-id:f802b234-60ca-4a65-bf73-21a0bd69d4d7,key:/events/openstack,resourceVersion:0,resourceVersionMatch:,limit:500,continue: 536ms (20:00:51.903)] Trace[1614859733]: [536.495442ms] [536.495442ms] END I0628 20:00:52.440033 1 trace.go:236] Trace[957765709]: "List" accept:application/vnd.kubernetes.protobuf;as=PartialObjectMetadataList;g=meta.k8s.io;v=v1,application/json;as=PartialObjectMetadataList;g=meta.k8s.io;v=v1,application/json,audit-id:19980a7d-1d18-4752-85a4-2a83d1f060ac,client:199.204.45.169,protocol:HTTP/2.0,resource:perconaxtradbclusterbackups,scope:cluster,url:/apis/pxc.percona.com/v1/perconaxtradbclusterbackups,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/metadata-informers,verb:LIST (28-Jun-2026 20:00:51.919) (total time: 520ms): Trace[957765709]: ["List(recursive=true) etcd3" audit-id:19980a7d-1d18-4752-85a4-2a83d1f060ac,key:/pxc.percona.com/perconaxtradbclusterbackups,resourceVersion:0,resourceVersionMatch:,limit:500,continue: 520ms (20:00:51.919)] Trace[957765709]: [520.753959ms] [520.753959ms] END I0628 20:00:52.441521 1 trace.go:236] Trace[1534694290]: "Get" accept:application/json,audit-id:13f68d46-f4c4-45ce-b021-e69749381e85,client:199.204.45.169,protocol:HTTP/1.1,resource:perconaxtradbclusters,scope:resource,url:/apis/pxc.percona.com/v1/namespaces/openstack/perconaxtradbclusters/percona-xtradb,user-agent:OpenAPI-Generator/36.0.2/python,verb:GET (28-Jun-2026 20:00:51.777) (total time: 663ms): Trace[1534694290]: ---"About to write a response" 662ms (20:00:52.440) Trace[1534694290]: [663.503474ms] [663.503474ms] END I0628 20:00:52.767418 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-cluster-operator" clusterIPs={"IPv4":"10.101.123.114"} I0628 20:00:53.070764 1 controller.go:624] quota admission added evaluator for: statefulsets.apps I0628 20:00:53.180016 1 controller.go:624] quota admission added evaluator for: poddisruptionbudgets.policy I0628 20:00:53.336513 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy" clusterIPs={"IPv4":"10.100.7.211"} I0628 20:00:53.352057 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy-replicas" clusterIPs={"IPv4":"10.109.228.187"} I0628 20:00:57.613329 1 trace.go:236] Trace[576007613]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:1ba717d9-8d51-401a-bd88-67de588b6728,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/local-path-storage/pods/helper-pod-create-pvc-27d20d20-a325-45df-9b27-ba04f9951b7e/status,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (28-Jun-2026 20:00:54.972) (total time: 2640ms): Trace[576007613]: ["GuaranteedUpdate etcd3" audit-id:1ba717d9-8d51-401a-bd88-67de588b6728,key:/pods/local-path-storage/helper-pod-create-pvc-27d20d20-a325-45df-9b27-ba04f9951b7e,type:*core.Pod,resource:pods 2640ms (20:00:54.973) Trace[576007613]: ---"Txn call completed" 2636ms (20:00:57.612)] Trace[576007613]: ---"Object stored in database" 2637ms (20:00:57.612) Trace[576007613]: [2.640232966s] [2.640232966s] END I0628 20:00:58.710801 1 trace.go:236] Trace[821052315]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:b282a3ff-ee82-4b38-a7e4-1f4fd987c0eb,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:PUT (28-Jun-2026 20:00:55.064) (total time: 3646ms): Trace[821052315]: ["GuaranteedUpdate etcd3" audit-id:b282a3ff-ee82-4b38-a7e4-1f4fd987c0eb,key:/leases/ingress-nginx/ingress-nginx-leader,type:*coordination.Lease,resource:leases.coordination.k8s.io 3646ms (20:00:55.064) Trace[821052315]: ---"Txn call completed" 3645ms (20:00:58.710)] Trace[821052315]: [3.646579906s] [3.646579906s] END I0628 20:00:58.711717 1 trace.go:236] Trace[2145970900]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:6f7fa95e-1234-42dd-9552-1cff3f7f13c0,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/local-path-storage/pods/helper-pod-create-pvc-27d20d20-a325-45df-9b27-ba04f9951b7e,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:00:57.617) (total time: 1094ms): Trace[2145970900]: ---"About to write a response" 1094ms (20:00:58.711) Trace[2145970900]: [1.094449867s] [1.094449867s] END I0628 20:00:58.711733 1 trace.go:236] Trace[572793129]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:d740362e-6eed-40b7-8b79-75881b0a81ab,client:10.0.0.187,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:00:56.526) (total time: 2184ms): Trace[572793129]: ["GuaranteedUpdate etcd3" audit-id:d740362e-6eed-40b7-8b79-75881b0a81ab,key:/leases/openstack/08db1feb.percona.com,type:*coordination.Lease,resource:leases.coordination.k8s.io 2184ms (20:00:56.527) Trace[572793129]: ---"Txn call completed" 2183ms (20:00:58.711)] Trace[572793129]: [2.184727643s] [2.184727643s] END I0628 20:00:58.711789 1 trace.go:236] Trace[822225222]: "Get" accept:application/json, */*,audit-id:30625d4a-cc3d-4ed9-ae96-3acdfb8d96f0,client:10.0.0.117,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-cainjector-leader-election,user-agent:cainjector/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:00:55.321) (total time: 3390ms): Trace[822225222]: ---"About to write a response" 3390ms (20:00:58.711) Trace[822225222]: [3.390588202s] [3.390588202s] END I0628 20:00:58.711884 1 trace.go:236] Trace[1662805660]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:254fdd84-c703-44fe-91f3-af40ec5f8f93,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Jun-2026 20:00:57.515) (total time: 1196ms): Trace[1662805660]: ["GuaranteedUpdate etcd3" audit-id:254fdd84-c703-44fe-91f3-af40ec5f8f93,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 1195ms (20:00:57.515) Trace[1662805660]: ---"Txn call completed" 1194ms (20:00:58.711)] Trace[1662805660]: [1.196043881s] [1.196043881s] END I0628 20:00:58.712010 1 trace.go:236] Trace[302346189]: "Get" accept:application/json, */*,audit-id:aaf9291b-7ef7-4056-ac90-427645164d25,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:00:56.461) (total time: 2250ms): Trace[302346189]: ---"About to write a response" 2250ms (20:00:58.711) Trace[302346189]: [2.250220382s] [2.250220382s] END I0628 20:00:58.711723 1 trace.go:236] Trace[83052227]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:2a1b1e1f-6d26-42be-a3f0-95d28ae279b0,client:10.0.0.170,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:00:56.452) (total time: 2258ms): Trace[83052227]: ["GuaranteedUpdate etcd3" audit-id:2a1b1e1f-6d26-42be-a3f0-95d28ae279b0,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 2258ms (20:00:56.452) Trace[83052227]: ---"Txn call completed" 2257ms (20:00:58.711)] Trace[83052227]: [2.2588968s] [2.2588968s] END I0628 20:00:58.712339 1 trace.go:236] Trace[596368845]: "Update" accept:application/json, */*,audit-id:027a994e-3104-4a3a-b744-11afe45e701e,client:10.0.0.173,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:00:56.451) (total time: 2260ms): Trace[596368845]: ["GuaranteedUpdate etcd3" audit-id:027a994e-3104-4a3a-b744-11afe45e701e,key:/leases/openstack/messaging-topology-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 2260ms (20:00:56.451) Trace[596368845]: ---"Txn call completed" 2258ms (20:00:58.711)] Trace[596368845]: [2.260324627s] [2.260324627s] END I0628 20:00:58.712782 1 trace.go:236] Trace[1812689567]: "Get" accept:application/json, */*,audit-id:054a90dc-fbf9-4c89-a01d-1a969fa9e399,client:10.0.0.139,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-controller,user-agent:cert-manager-leader-election/v1.12.17 (linux/amd64) cert-manager/37b853ff34a6c093e946c657c189f40413c0f628,verb:GET (28-Jun-2026 20:00:57.781) (total time: 931ms): Trace[1812689567]: ---"About to write a response" 931ms (20:00:58.712) Trace[1812689567]: [931.656955ms] [931.656955ms] END I0628 20:00:58.712822 1 trace.go:236] Trace[984888167]: "Get" accept:application/json, */*,audit-id:ff1f6e25-0abe-4c17-8dae-371329281592,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:00:56.461) (total time: 2251ms): Trace[984888167]: ---"About to write a response" 2250ms (20:00:58.712) Trace[984888167]: [2.251028608s] [2.251028608s] END I0628 20:00:58.712839 1 trace.go:236] Trace[831607996]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:8993c12c-0d1b-43ba-a36f-e4c257566c72,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:00:55.468) (total time: 3244ms): Trace[831607996]: ---"About to write a response" 3244ms (20:00:58.712) Trace[831607996]: [3.244270557s] [3.244270557s] END I0628 20:00:58.712862 1 trace.go:236] Trace[1821300061]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:184ee459-9cdb-414b-9eca-b9bcf3069ccc,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:00:56.459) (total time: 2253ms): Trace[1821300061]: ---"About to write a response" 2253ms (20:00:58.712) Trace[1821300061]: [2.253241681s] [2.253241681s] END I0628 20:00:58.712984 1 trace.go:236] Trace[967735322]: "Get" accept:application/json, */*,audit-id:70d44b92-6251-4fc9-8dde-c246846a6677,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:00:56.456) (total time: 2255ms): Trace[967735322]: ---"About to write a response" 2255ms (20:00:58.712) Trace[967735322]: [2.255988124s] [2.255988124s] END I0628 20:00:58.713161 1 trace.go:236] Trace[2088700333]: "Get" accept:application/json,audit-id:42b0f7d2-2726-455f-b7ea-7b37ae040e06,client:199.204.45.169,protocol:HTTP/1.1,resource:perconaxtradbclusters,scope:resource,url:/apis/pxc.percona.com/v1/namespaces/openstack/perconaxtradbclusters/percona-xtradb,user-agent:OpenAPI-Generator/36.0.2/python,verb:GET (28-Jun-2026 20:00:55.459) (total time: 3253ms): Trace[2088700333]: ---"About to write a response" 3252ms (20:00:58.712) Trace[2088700333]: [3.253166238s] [3.253166238s] END I0628 20:00:58.713213 1 trace.go:236] Trace[1547811578]: "Get" accept:application/json, */*,audit-id:39e7ca27-bd8d-4f8c-90ba-ca0544c8aca5,client:10.0.0.25,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:00:56.462) (total time: 2250ms): Trace[1547811578]: ---"About to write a response" 2250ms (20:00:58.712) Trace[1547811578]: [2.250845954s] [2.250845954s] END I0628 20:00:58.714870 1 trace.go:236] Trace[762721472]: "Get" accept:application/json, */*,audit-id:c045086a-e043-4908-be64-91de13d32a27,client:10.0.0.193,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/local-path-storage/pods/helper-pod-create-pvc-27d20d20-a325-45df-9b27-ba04f9951b7e,user-agent:local-path-provisioner/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:00:55.167) (total time: 3547ms): Trace[762721472]: ---"About to write a response" 3545ms (20:00:58.713) Trace[762721472]: [3.54700545s] [3.54700545s] END I0628 20:00:58.772419 1 trace.go:236] Trace[1401457501]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.169,type:*v1.Endpoints,resource:apiServerIPInfo (28-Jun-2026 20:00:55.323) (total time: 3449ms): Trace[1401457501]: ---"initial value restored" 3388ms (20:00:58.711) Trace[1401457501]: [3.449076146s] [3.449076146s] END I0628 20:01:44.135487 1 trace.go:236] Trace[801798870]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:55c3ddf7-3729-4c75-a0c1-0c78f83c2c8e,client:199.204.45.169,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/openstack/events/percona-xtradb-haproxy-0.18bd572f656aa0a8,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (28-Jun-2026 20:01:43.582) (total time: 553ms): Trace[801798870]: ["GuaranteedUpdate etcd3" audit-id:55c3ddf7-3729-4c75-a0c1-0c78f83c2c8e,key:/events/openstack/percona-xtradb-haproxy-0.18bd572f656aa0a8,type:*core.Event,resource:events 552ms (20:01:43.582) Trace[801798870]: ---"Txn call completed" 520ms (20:01:44.135)] Trace[801798870]: ---"Object stored in database" 521ms (20:01:44.135) Trace[801798870]: [553.0465ms] [553.0465ms] END I0628 20:01:44.156399 1 trace.go:236] Trace[110818079]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:3fbad5bb-f7d8-43dc-84f1-db223f312d37,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (28-Jun-2026 20:01:43.615) (total time: 541ms): Trace[110818079]: ["GuaranteedUpdate etcd3" audit-id:3fbad5bb-f7d8-43dc-84f1-db223f312d37,key:/leases/kube-system/kube-scheduler,type:*coordination.Lease,resource:leases.coordination.k8s.io 541ms (20:01:43.615) Trace[110818079]: ---"Txn call completed" 540ms (20:01:44.156)] Trace[110818079]: [541.114402ms] [541.114402ms] END I0628 20:01:44.156423 1 trace.go:236] Trace[1365096619]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:067f5fd8-6299-4021-a85d-fe92018ad948,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (28-Jun-2026 20:01:43.615) (total time: 541ms): Trace[1365096619]: ["GuaranteedUpdate etcd3" audit-id:067f5fd8-6299-4021-a85d-fe92018ad948,key:/leases/kube-system/kube-controller-manager,type:*coordination.Lease,resource:leases.coordination.k8s.io 541ms (20:01:43.615) Trace[1365096619]: ---"Txn call completed" 540ms (20:01:44.156)] Trace[1365096619]: [541.140822ms] [541.140822ms] END I0628 20:01:44.158200 1 trace.go:236] Trace[1824362529]: "Update" accept:application/json, */*,audit-id:5284b1d1-3f5e-464a-ad4c-f9fe0edd10b2,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:01:43.615) (total time: 542ms): Trace[1824362529]: ["GuaranteedUpdate etcd3" audit-id:5284b1d1-3f5e-464a-ad4c-f9fe0edd10b2,key:/leases/kube-system/cilium-operator-resource-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 542ms (20:01:43.615) Trace[1824362529]: ---"Txn call completed" 541ms (20:01:44.157)] Trace[1824362529]: [542.487408ms] [542.487408ms] END I0628 20:01:44.158299 1 trace.go:236] Trace[1118863795]: "Update" accept:application/json, */*,audit-id:ff027be9-f734-475e-b12f-455cc120e9dc,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Jun-2026 20:01:43.616) (total time: 541ms): Trace[1118863795]: ["GuaranteedUpdate etcd3" audit-id:ff027be9-f734-475e-b12f-455cc120e9dc,key:/leases/kube-system/plndr-svcs-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 541ms (20:01:43.616) Trace[1118863795]: ---"Txn call completed" 540ms (20:01:44.158)] Trace[1118863795]: [541.937358ms] [541.937358ms] END I0628 20:01:44.158534 1 trace.go:236] Trace[976535953]: "Update" accept:application/json, */*,audit-id:bea7e6e1-78bf-484c-be30-223c95f85f84,client:10.0.0.25,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:01:43.615) (total time: 542ms): Trace[976535953]: ["GuaranteedUpdate etcd3" audit-id:bea7e6e1-78bf-484c-be30-223c95f85f84,key:/leases/openstack/rabbitmq-cluster-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 542ms (20:01:43.615) Trace[976535953]: ---"Txn call completed" 541ms (20:01:44.158)] Trace[976535953]: [542.836115ms] [542.836115ms] END I0628 20:01:44.159106 1 trace.go:236] Trace[49293866]: "Update" accept:application/json, */*,audit-id:bed1b70a-2329-4f89-aceb-f96271a9da71,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Jun-2026 20:01:43.616) (total time: 542ms): Trace[49293866]: ["GuaranteedUpdate etcd3" audit-id:bed1b70a-2329-4f89-aceb-f96271a9da71,key:/leases/kube-system/plndr-cp-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 542ms (20:01:43.616) Trace[49293866]: ---"Txn call completed" 540ms (20:01:44.158)] Trace[49293866]: [542.420616ms] [542.420616ms] END I0628 20:01:44.952369 1 trace.go:236] Trace[2138419585]: "Update" accept:application/json, */*,audit-id:c4a39f7f-5aa2-497a-b3a4-0ecc9a29a069,client:10.0.0.117,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-cainjector-leader-election,user-agent:cainjector/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:01:44.232) (total time: 720ms): Trace[2138419585]: ["GuaranteedUpdate etcd3" audit-id:c4a39f7f-5aa2-497a-b3a4-0ecc9a29a069,key:/leases/cert-manager/cert-manager-cainjector-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 719ms (20:01:44.232) Trace[2138419585]: ---"Txn call completed" 718ms (20:01:44.951)] Trace[2138419585]: [720.003242ms] [720.003242ms] END I0628 20:01:44.952920 1 trace.go:236] Trace[819383311]: "Update" accept:application/json, */*,audit-id:efc5d0ea-1bf7-43e8-bab3-cb7cb86106ca,client:10.0.0.139,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-controller,user-agent:cert-manager-leader-election/v1.12.17 (linux/amd64) cert-manager/37b853ff34a6c093e946c657c189f40413c0f628,verb:PUT (28-Jun-2026 20:01:44.231) (total time: 721ms): Trace[819383311]: ["GuaranteedUpdate etcd3" audit-id:efc5d0ea-1bf7-43e8-bab3-cb7cb86106ca,key:/leases/cert-manager/cert-manager-controller,type:*coordination.Lease,resource:leases.coordination.k8s.io 720ms (20:01:44.232) Trace[819383311]: ---"Txn call completed" 718ms (20:01:44.951)] Trace[819383311]: [721.071682ms] [721.071682ms] END I0628 20:01:44.953661 1 trace.go:236] Trace[1258800026]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:d1e52f3f-582d-4124-8c00-1646afae67a3,client:10.0.0.187,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/percona-xtradb-pxc-0,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:01:44.377) (total time: 576ms): Trace[1258800026]: ---"About to write a response" 575ms (20:01:44.953) Trace[1258800026]: [576.183071ms] [576.183071ms] END I0628 20:01:47.393800 1 trace.go:236] Trace[1694597970]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.169,type:*v1.Endpoints,resource:apiServerIPInfo (28-Jun-2026 20:01:45.325) (total time: 2067ms): Trace[1694597970]: ---"Transaction prepared" 583ms (20:01:45.911) Trace[1694597970]: ---"Txn call completed" 1482ms (20:01:47.393) Trace[1694597970]: [2.067733205s] [2.067733205s] END I0628 20:01:47.395069 1 trace.go:236] Trace[2096157409]: "Get" accept:application/json, */*,audit-id:e648c1e0-274a-422a-a140-3223cee2ec10,client:10.0.0.25,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:01:46.161) (total time: 1233ms): Trace[2096157409]: ---"About to write a response" 1233ms (20:01:47.394) Trace[2096157409]: [1.233268483s] [1.233268483s] END I0628 20:01:47.395143 1 trace.go:236] Trace[646265555]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:03f7ed1a-1f47-442e-98a0-534fc6d693b2,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:01:46.159) (total time: 1235ms): Trace[646265555]: ---"About to write a response" 1235ms (20:01:47.394) Trace[646265555]: [1.235553556s] [1.235553556s] END I0628 20:01:47.395328 1 trace.go:236] Trace[2008467868]: "Get" accept:application/json, */*,audit-id:e28478e4-3961-4baa-8cce-36e6ba3f034f,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:01:46.161) (total time: 1234ms): Trace[2008467868]: ---"About to write a response" 1233ms (20:01:47.395) Trace[2008467868]: [1.234022827s] [1.234022827s] END I0628 20:01:47.395350 1 trace.go:236] Trace[963286696]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:0ca199a7-6a26-4bcb-b444-e6670f7a3cbe,client:10.0.0.187,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:01:45.593) (total time: 1801ms): Trace[963286696]: ["GuaranteedUpdate etcd3" audit-id:0ca199a7-6a26-4bcb-b444-e6670f7a3cbe,key:/leases/openstack/08db1feb.percona.com,type:*coordination.Lease,resource:leases.coordination.k8s.io 1801ms (20:01:45.593) Trace[963286696]: ---"Txn call completed" 1799ms (20:01:47.394)] Trace[963286696]: [1.801702385s] [1.801702385s] END I0628 20:01:47.395354 1 trace.go:236] Trace[530966945]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:886251a7-402f-4c75-a79c-0e9310e93111,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:01:46.160) (total time: 1235ms): Trace[530966945]: ---"About to write a response" 1234ms (20:01:47.395) Trace[530966945]: [1.235139888s] [1.235139888s] END I0628 20:01:47.395354 1 trace.go:236] Trace[117923323]: "Get" accept:application/json, */*,audit-id:4e4b264a-d492-4c70-acc4-4d57ec3e091d,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:01:46.160) (total time: 1234ms): Trace[117923323]: ---"About to write a response" 1234ms (20:01:47.395) Trace[117923323]: [1.23469732s] [1.23469732s] END I0628 20:01:47.395628 1 trace.go:236] Trace[1117133217]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:044f86a7-54e2-4e2b-8f29-4a3f2b314319,client:10.0.0.170,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:01:45.616) (total time: 1779ms): Trace[1117133217]: ["GuaranteedUpdate etcd3" audit-id:044f86a7-54e2-4e2b-8f29-4a3f2b314319,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 1779ms (20:01:45.616) Trace[1117133217]: ---"Txn call completed" 1777ms (20:01:47.394)] Trace[1117133217]: [1.779308649s] [1.779308649s] END I0628 20:01:47.395905 1 trace.go:236] Trace[1421267672]: "Update" accept:application/json, */*,audit-id:6fd2dcb0-ea45-4cee-a512-3229570d4425,client:10.0.0.173,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:01:45.616) (total time: 1779ms): Trace[1421267672]: ["GuaranteedUpdate etcd3" audit-id:6fd2dcb0-ea45-4cee-a512-3229570d4425,key:/leases/openstack/messaging-topology-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 1779ms (20:01:45.616) Trace[1421267672]: ---"Txn call completed" 1778ms (20:01:47.395)] Trace[1421267672]: [1.779662116s] [1.779662116s] END I0628 20:01:47.396209 1 trace.go:236] Trace[945999781]: "Get" accept:application/json, */*,audit-id:6972193f-b6f4-4b39-99de-85b542961b57,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:01:46.161) (total time: 1234ms): Trace[945999781]: ---"About to write a response" 1234ms (20:01:47.395) Trace[945999781]: [1.234463456s] [1.234463456s] END I0628 20:01:47.396805 1 trace.go:236] Trace[957147809]: "Get" accept:application/json,audit-id:8cee441c-0085-4962-9952-2b870ad31230,client:199.204.45.169,protocol:HTTP/1.1,resource:perconaxtradbclusters,scope:resource,url:/apis/pxc.percona.com/v1/namespaces/openstack/perconaxtradbclusters/percona-xtradb,user-agent:OpenAPI-Generator/36.0.2/python,verb:GET (28-Jun-2026 20:01:45.958) (total time: 1437ms): Trace[957147809]: ---"About to write a response" 1437ms (20:01:47.396) Trace[957147809]: [1.437979615s] [1.437979615s] END I0628 20:01:48.550432 1 trace.go:236] Trace[777634312]: "List(recursive=true) etcd3" audit-id:,key:/masterleases/,resourceVersion:0,resourceVersionMatch:NotOlderThan,limit:0,continue: (28-Jun-2026 20:01:47.405) (total time: 1144ms): Trace[777634312]: [1.144921668s] [1.144921668s] END I0628 20:01:48.550783 1 trace.go:236] Trace[1648342993]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:59b7d292-30fe-418d-8837-de95aa5826b4,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (28-Jun-2026 20:01:47.397) (total time: 1153ms): Trace[1648342993]: ["GuaranteedUpdate etcd3" audit-id:59b7d292-30fe-418d-8837-de95aa5826b4,key:/leases/kube-system/kube-controller-manager,type:*coordination.Lease,resource:leases.coordination.k8s.io 1153ms (20:01:47.397) Trace[1648342993]: ---"Txn call completed" 1152ms (20:01:48.550)] Trace[1648342993]: [1.153209988s] [1.153209988s] END I0628 20:01:48.551079 1 trace.go:236] Trace[106603400]: "Update" accept:application/json, */*,audit-id:f4796d2d-2490-43f1-af83-43def008d5cd,client:10.0.0.25,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:01:47.397) (total time: 1153ms): Trace[106603400]: ["GuaranteedUpdate etcd3" audit-id:f4796d2d-2490-43f1-af83-43def008d5cd,key:/leases/openstack/rabbitmq-cluster-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 1153ms (20:01:47.397) Trace[106603400]: ---"Txn call completed" 1152ms (20:01:48.550)] Trace[106603400]: [1.153656116s] [1.153656116s] END I0628 20:01:48.551353 1 trace.go:236] Trace[226356882]: "Update" accept:application/json, */*,audit-id:2074d0ef-7de3-41f2-82ff-8604ad4d86f3,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Jun-2026 20:01:47.398) (total time: 1153ms): Trace[226356882]: ["GuaranteedUpdate etcd3" audit-id:2074d0ef-7de3-41f2-82ff-8604ad4d86f3,key:/leases/kube-system/plndr-svcs-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 1152ms (20:01:47.398) Trace[226356882]: ---"Txn call completed" 1152ms (20:01:48.551)] Trace[226356882]: [1.153067524s] [1.153067524s] END I0628 20:01:48.551394 1 trace.go:236] Trace[125718759]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:b69cc5d4-443d-4411-99d3-f4ea89274d52,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (28-Jun-2026 20:01:47.397) (total time: 1154ms): Trace[125718759]: ["GuaranteedUpdate etcd3" audit-id:b69cc5d4-443d-4411-99d3-f4ea89274d52,key:/leases/kube-system/kube-scheduler,type:*coordination.Lease,resource:leases.coordination.k8s.io 1154ms (20:01:47.397) Trace[125718759]: ---"Txn call completed" 1153ms (20:01:48.551)] Trace[125718759]: [1.154256346s] [1.154256346s] END I0628 20:01:48.551725 1 trace.go:236] Trace[470679218]: "Update" accept:application/json, */*,audit-id:0a353225-93ab-4736-9941-c46e1bde3642,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Jun-2026 20:01:47.397) (total time: 1154ms): Trace[470679218]: ["GuaranteedUpdate etcd3" audit-id:0a353225-93ab-4736-9941-c46e1bde3642,key:/leases/kube-system/plndr-cp-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 1154ms (20:01:47.397) Trace[470679218]: ---"Txn call completed" 1153ms (20:01:48.551)] Trace[470679218]: [1.154485481s] [1.154485481s] END I0628 20:01:48.552047 1 trace.go:236] Trace[433748809]: "Update" accept:application/json, */*,audit-id:b0ca7f71-c6b7-4e95-8080-f2cd2e05b014,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:01:47.397) (total time: 1154ms): Trace[433748809]: ["GuaranteedUpdate etcd3" audit-id:b0ca7f71-c6b7-4e95-8080-f2cd2e05b014,key:/leases/kube-system/cilium-operator-resource-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 1154ms (20:01:47.397) Trace[433748809]: ---"Txn call completed" 1153ms (20:01:48.551)] Trace[433748809]: [1.154799587s] [1.154799587s] END I0628 20:02:25.586052 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy-metrics" clusterIPs={"IPv4":"10.101.237.206"} I0628 20:02:28.216046 1 alloc.go:330] "allocated clusterIPs" service="openstack/valkey" clusterIPs={"IPv4":"10.110.171.248"} I0628 20:02:28.224778 1 alloc.go:330] "allocated clusterIPs" service="openstack/valkey-metrics" clusterIPs={"IPv4":"10.101.236.164"} I0628 20:02:35.659789 1 alloc.go:330] "allocated clusterIPs" service="auth-system/keycloak-metrics" clusterIPs={"IPv4":"10.98.189.129"} I0628 20:02:35.670793 1 alloc.go:330] "allocated clusterIPs" service="auth-system/keycloak" clusterIPs={"IPv4":"10.99.7.207"} I0628 20:02:40.253451 1 trace.go:236] Trace[241568427]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:ea1ed432-ffce-4efe-af87-ac77a1f3b58a,client:10.0.0.170,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:02:39.640) (total time: 612ms): Trace[241568427]: ["GuaranteedUpdate etcd3" audit-id:ea1ed432-ffce-4efe-af87-ac77a1f3b58a,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 612ms (20:02:39.640) Trace[241568427]: ---"Txn call completed" 611ms (20:02:40.253)] Trace[241568427]: [612.721998ms] [612.721998ms] END I0628 20:02:40.259125 1 trace.go:236] Trace[980191528]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:7c6cfe44-fb12-403f-83f3-53559008fc43,client:10.0.0.187,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:02:39.641) (total time: 618ms): Trace[980191528]: ["GuaranteedUpdate etcd3" audit-id:7c6cfe44-fb12-403f-83f3-53559008fc43,key:/leases/openstack/08db1feb.percona.com,type:*coordination.Lease,resource:leases.coordination.k8s.io 617ms (20:02:39.641) Trace[980191528]: ---"Txn call completed" 616ms (20:02:40.258)] Trace[980191528]: [618.02278ms] [618.02278ms] END I0628 20:02:40.259316 1 trace.go:236] Trace[53051174]: "Get" accept:application/json, */*,audit-id:d2340dcb-f478-44ff-9f61-0820b61abd64,client:199.204.45.169,protocol:HTTP/2.0,resource:services,scope:resource,url:/api/v1/namespaces/auth-system/services/keycloak-headless,user-agent:Helm/3.11.2,verb:GET (28-Jun-2026 20:02:39.750) (total time: 508ms): Trace[53051174]: ---"About to write a response" 508ms (20:02:40.259) Trace[53051174]: [508.324573ms] [508.324573ms] END I0628 20:02:40.259503 1 trace.go:236] Trace[500283848]: "Update" accept:application/json, */*,audit-id:df8660db-1859-4306-833b-fd9d85e2b39a,client:10.0.0.173,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:02:39.656) (total time: 603ms): Trace[500283848]: ["GuaranteedUpdate etcd3" audit-id:df8660db-1859-4306-833b-fd9d85e2b39a,key:/leases/openstack/messaging-topology-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 602ms (20:02:39.656) Trace[500283848]: ---"Txn call completed" 601ms (20:02:40.259)] Trace[500283848]: [603.327088ms] [603.327088ms] END I0628 20:03:07.691534 1 trace.go:236] Trace[2027611227]: "Get" accept:application/json, */*,audit-id:f2e7c94d-9910-4609-ac02-e8b86537555b,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:03:07.089) (total time: 601ms): Trace[2027611227]: ---"About to write a response" 601ms (20:03:07.691) Trace[2027611227]: [601.93388ms] [601.93388ms] END I0628 20:03:07.691658 1 trace.go:236] Trace[12791749]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:50ceb79c-3c9d-453b-9234-880b11503d9d,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:PUT (28-Jun-2026 20:03:06.819) (total time: 871ms): Trace[12791749]: ["GuaranteedUpdate etcd3" audit-id:50ceb79c-3c9d-453b-9234-880b11503d9d,key:/leases/ingress-nginx/ingress-nginx-leader,type:*coordination.Lease,resource:leases.coordination.k8s.io 871ms (20:03:06.819) Trace[12791749]: ---"Txn call completed" 870ms (20:03:07.691)] Trace[12791749]: [871.998357ms] [871.998357ms] END I0628 20:03:07.691921 1 trace.go:236] Trace[725139344]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:1d24a057-b3b0-404b-bea4-d0cad2963fe8,client:10.0.0.187,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:03:06.901) (total time: 790ms): Trace[725139344]: ["GuaranteedUpdate etcd3" audit-id:1d24a057-b3b0-404b-bea4-d0cad2963fe8,key:/leases/openstack/08db1feb.percona.com,type:*coordination.Lease,resource:leases.coordination.k8s.io 790ms (20:03:06.901) Trace[725139344]: ---"Txn call completed" 789ms (20:03:07.691)] Trace[725139344]: [790.475452ms] [790.475452ms] END I0628 20:03:07.691927 1 trace.go:236] Trace[166684436]: "Get" accept:application/json, */*,audit-id:f6d40e5b-4242-4d46-86cb-47183ce3daa5,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:03:07.077) (total time: 613ms): Trace[166684436]: ---"About to write a response" 613ms (20:03:07.691) Trace[166684436]: [613.625325ms] [613.625325ms] END I0628 20:03:07.691997 1 trace.go:236] Trace[85715407]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:81cfcb29-c865-4ad6-8da1-29f51d2d53a2,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:03:07.078) (total time: 613ms): Trace[85715407]: ---"About to write a response" 613ms (20:03:07.691) Trace[85715407]: [613.755258ms] [613.755258ms] END I0628 20:03:07.692214 1 trace.go:236] Trace[1486271278]: "Update" accept:application/json, */*,audit-id:41a4f9b4-c17f-4bd8-897a-eaf2b3c85f49,client:10.0.0.173,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:03:06.904) (total time: 787ms): Trace[1486271278]: ["GuaranteedUpdate etcd3" audit-id:41a4f9b4-c17f-4bd8-897a-eaf2b3c85f49,key:/leases/openstack/messaging-topology-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 787ms (20:03:06.904) Trace[1486271278]: ---"Txn call completed" 786ms (20:03:07.692)] Trace[1486271278]: [787.7588ms] [787.7588ms] END I0628 20:03:07.692349 1 trace.go:236] Trace[1278574908]: "Get" accept:application/json, */*,audit-id:9acc7144-0248-49f8-ac05-fe7095d19cbc,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:03:07.086) (total time: 605ms): Trace[1278574908]: ---"About to write a response" 605ms (20:03:07.692) Trace[1278574908]: [605.863546ms] [605.863546ms] END I0628 20:03:07.692231 1 trace.go:236] Trace[1467164167]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:6f434765-19f3-439d-b391-dfc82ca9e0b5,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:03:07.080) (total time: 611ms): Trace[1467164167]: ---"About to write a response" 611ms (20:03:07.692) Trace[1467164167]: [611.517065ms] [611.517065ms] END I0628 20:03:07.692932 1 trace.go:236] Trace[18999719]: "Get" accept:application/json, */*,audit-id:e663ad2c-8427-4ef5-b8ad-e9d65ca4808d,client:10.0.0.25,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:03:07.085) (total time: 607ms): Trace[18999719]: ---"About to write a response" 607ms (20:03:07.692) Trace[18999719]: [607.829224ms] [607.829224ms] END I0628 20:03:15.996113 1 trace.go:236] Trace[1892643543]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:0d461a91-681f-48fa-a2a6-44cab6c73677,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Jun-2026 20:03:13.728) (total time: 2267ms): Trace[1892643543]: ["GuaranteedUpdate etcd3" audit-id:0d461a91-681f-48fa-a2a6-44cab6c73677,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 2267ms (20:03:13.728) Trace[1892643543]: ---"Txn call completed" 2266ms (20:03:15.995)] Trace[1892643543]: [2.267587352s] [2.267587352s] END I0628 20:03:15.996549 1 trace.go:236] Trace[472041650]: "Get" accept:application/json, */*,audit-id:38467e61-1a76-41c7-b629-2653f9acecc5,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:03:13.739) (total time: 2256ms): Trace[472041650]: ---"About to write a response" 2256ms (20:03:15.996) Trace[472041650]: [2.256509829s] [2.256509829s] END I0628 20:03:16.002370 1 trace.go:236] Trace[91441147]: "Update" accept:application/json, */*,audit-id:c8eacf23-0c3d-4cfc-9388-6520183767e8,client:10.0.0.173,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:03:13.713) (total time: 2288ms): Trace[91441147]: ["GuaranteedUpdate etcd3" audit-id:c8eacf23-0c3d-4cfc-9388-6520183767e8,key:/leases/openstack/messaging-topology-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 2287ms (20:03:13.715) Trace[91441147]: ---"Txn call completed" 2278ms (20:03:16.002)] Trace[91441147]: [2.288504854s] [2.288504854s] END I0628 20:03:16.002925 1 trace.go:236] Trace[224554481]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:b89a6f4c-ade1-4406-adec-cf4ec52b9211,client:10.0.0.187,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:03:13.712) (total time: 2290ms): Trace[224554481]: ["GuaranteedUpdate etcd3" audit-id:b89a6f4c-ade1-4406-adec-cf4ec52b9211,key:/leases/openstack/08db1feb.percona.com,type:*coordination.Lease,resource:leases.coordination.k8s.io 2290ms (20:03:13.712) Trace[224554481]: ---"Txn call completed" 2287ms (20:03:16.002)] Trace[224554481]: [2.290491723s] [2.290491723s] END I0628 20:03:16.022396 1 trace.go:236] Trace[2052055651]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:18a6247f-867a-4cef-aa7a-008a5afe2c92,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:PUT (28-Jun-2026 20:03:15.194) (total time: 827ms): Trace[2052055651]: ["GuaranteedUpdate etcd3" audit-id:18a6247f-867a-4cef-aa7a-008a5afe2c92,key:/leases/ingress-nginx/ingress-nginx-leader,type:*coordination.Lease,resource:leases.coordination.k8s.io 827ms (20:03:15.195) Trace[2052055651]: ---"Txn call completed" 826ms (20:03:16.022)] Trace[2052055651]: [827.454302ms] [827.454302ms] END I0628 20:03:16.023573 1 trace.go:236] Trace[486097321]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:a0f7ed74-6b8e-4435-a944-497972998f36,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:03:13.756) (total time: 2267ms): Trace[486097321]: ---"About to write a response" 2267ms (20:03:16.023) Trace[486097321]: [2.267454179s] [2.267454179s] END I0628 20:03:16.023744 1 trace.go:236] Trace[1094185672]: "Get" accept:application/json, */*,audit-id:be399562-c79f-4680-8b69-57f1097b1ecc,client:199.204.45.169,protocol:HTTP/2.0,resource:services,scope:resource,url:/api/v1/namespaces/auth-system/services/keycloak-headless,user-agent:Helm/3.11.2,verb:GET (28-Jun-2026 20:03:13.750) (total time: 2273ms): Trace[1094185672]: ---"About to write a response" 2273ms (20:03:16.023) Trace[1094185672]: [2.273538235s] [2.273538235s] END I0628 20:03:16.023821 1 trace.go:236] Trace[1361901344]: "Get" accept:application/json, */*,audit-id:46ecb1cf-0796-4898-925a-7b0e9eb100c1,client:10.0.0.117,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-cainjector-leader-election,user-agent:cainjector/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:03:15.012) (total time: 1010ms): Trace[1361901344]: ---"About to write a response" 1010ms (20:03:16.023) Trace[1361901344]: [1.010915866s] [1.010915866s] END I0628 20:03:16.024141 1 trace.go:236] Trace[1253652840]: "Get" accept:application/json, */*,audit-id:fb01ea78-3649-4788-a43a-933026e51e92,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:03:13.760) (total time: 2263ms): Trace[1253652840]: ---"About to write a response" 2263ms (20:03:16.023) Trace[1253652840]: [2.263702127s] [2.263702127s] END I0628 20:03:16.024397 1 trace.go:236] Trace[1123915591]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:e02e3a6c-270a-4545-9682-aabea2ab9c24,client:10.0.0.170,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:03:15.337) (total time: 686ms): Trace[1123915591]: ["GuaranteedUpdate etcd3" audit-id:e02e3a6c-270a-4545-9682-aabea2ab9c24,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 686ms (20:03:15.338) Trace[1123915591]: ---"Txn call completed" 685ms (20:03:16.024)] Trace[1123915591]: [686.537495ms] [686.537495ms] END I0628 20:03:16.024415 1 trace.go:236] Trace[1656430625]: "Get" accept:application/json, */*,audit-id:9cdcca3f-b295-42cf-a77d-afff09282ee9,client:10.0.0.25,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:03:13.762) (total time: 2262ms): Trace[1656430625]: ---"About to write a response" 2261ms (20:03:16.024) Trace[1656430625]: [2.262206539s] [2.262206539s] END I0628 20:03:16.024562 1 trace.go:236] Trace[606636132]: "Get" accept:application/json, */*,audit-id:6390091d-99c4-4485-89cf-50823499125e,client:10.0.0.139,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-controller,user-agent:cert-manager-leader-election/v1.12.17 (linux/amd64) cert-manager/37b853ff34a6c093e946c657c189f40413c0f628,verb:GET (28-Jun-2026 20:03:15.019) (total time: 1005ms): Trace[606636132]: ---"About to write a response" 1005ms (20:03:16.024) Trace[606636132]: [1.005381999s] [1.005381999s] END I0628 20:03:16.024623 1 trace.go:236] Trace[1789989444]: "Get" accept:application/json, */*,audit-id:106cf695-6f7c-41f5-aa40-d6d2262aa792,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:03:13.759) (total time: 2264ms): Trace[1789989444]: ---"About to write a response" 2264ms (20:03:16.024) Trace[1789989444]: [2.264666825s] [2.264666825s] END I0628 20:03:16.024767 1 trace.go:236] Trace[880882742]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:36aa317e-24e9-47f9-8285-22b5f615a5e8,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:03:13.762) (total time: 2262ms): Trace[880882742]: ---"About to write a response" 2262ms (20:03:16.024) Trace[880882742]: [2.262308031s] [2.262308031s] END I0628 20:03:17.066332 1 trace.go:236] Trace[1335584069]: "Update" accept:application/json, */*,audit-id:6cf0a8e9-5f76-433c-9998-14191acef261,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Jun-2026 20:03:16.002) (total time: 1063ms): Trace[1335584069]: ["GuaranteedUpdate etcd3" audit-id:6cf0a8e9-5f76-433c-9998-14191acef261,key:/leases/kube-system/plndr-svcs-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 1063ms (20:03:16.002) Trace[1335584069]: ---"Txn call completed" 1062ms (20:03:17.065)] Trace[1335584069]: [1.063751631s] [1.063751631s] END I0628 20:03:17.068187 1 trace.go:236] Trace[82303324]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:3e1d3c02-0e3a-4403-8cbc-b2ca12038f40,client:199.204.45.169,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/openstack/events/valkey-node-0.18bd574598087831,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (28-Jun-2026 20:03:14.123) (total time: 2944ms): Trace[82303324]: ["GuaranteedUpdate etcd3" audit-id:3e1d3c02-0e3a-4403-8cbc-b2ca12038f40,key:/events/openstack/valkey-node-0.18bd574598087831,type:*core.Event,resource:events 2944ms (20:03:14.123) Trace[82303324]: ---"initial value restored" 1900ms (20:03:16.024) Trace[82303324]: ---"Txn call completed" 1042ms (20:03:17.067)] Trace[82303324]: ---"Object stored in database" 1042ms (20:03:17.067) Trace[82303324]: [2.944498303s] [2.944498303s] END I0628 20:03:17.068215 1 trace.go:236] Trace[833296820]: "Update" accept:application/json, */*,audit-id:b4e76d98-1fc9-48a0-b1d3-654308f474cd,client:10.0.0.25,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:03:16.028) (total time: 1040ms): Trace[833296820]: ["GuaranteedUpdate etcd3" audit-id:b4e76d98-1fc9-48a0-b1d3-654308f474cd,key:/leases/openstack/rabbitmq-cluster-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 1039ms (20:03:16.028) Trace[833296820]: ---"Txn call completed" 1039ms (20:03:17.068)] Trace[833296820]: [1.040107336s] [1.040107336s] END I0628 20:03:17.068320 1 trace.go:236] Trace[437186608]: "Update" accept:application/json, */*,audit-id:a0088416-1364-4d0c-822f-d646cc0721e6,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Jun-2026 20:03:16.026) (total time: 1041ms): Trace[437186608]: ["GuaranteedUpdate etcd3" audit-id:a0088416-1364-4d0c-822f-d646cc0721e6,key:/leases/kube-system/plndr-cp-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 1041ms (20:03:16.027) Trace[437186608]: ---"Txn call completed" 1040ms (20:03:17.068)] Trace[437186608]: [1.041412012s] [1.041412012s] END I0628 20:03:17.068324 1 trace.go:236] Trace[858435052]: "Update" accept:application/json, */*,audit-id:801a1b90-19f3-42cf-a1ca-f7114cccb1d9,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:03:16.026) (total time: 1041ms): Trace[858435052]: ["GuaranteedUpdate etcd3" audit-id:801a1b90-19f3-42cf-a1ca-f7114cccb1d9,key:/leases/kube-system/cilium-operator-resource-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 1041ms (20:03:16.026) Trace[858435052]: ---"Txn call completed" 1040ms (20:03:17.068)] Trace[858435052]: [1.041573734s] [1.041573734s] END I0628 20:03:17.068285 1 trace.go:236] Trace[1326054496]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:e4718837-088f-4964-915a-df4f8093d409,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (28-Jun-2026 20:03:16.027) (total time: 1040ms): Trace[1326054496]: ["GuaranteedUpdate etcd3" audit-id:e4718837-088f-4964-915a-df4f8093d409,key:/leases/kube-system/kube-controller-manager,type:*coordination.Lease,resource:leases.coordination.k8s.io 1040ms (20:03:16.027) Trace[1326054496]: ---"Txn call completed" 1039ms (20:03:17.068)] Trace[1326054496]: [1.040948832s] [1.040948832s] END I0628 20:03:17.068439 1 trace.go:236] Trace[1896351420]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:455f6ddc-6a3d-498a-a08d-d5230dccad5c,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (28-Jun-2026 20:03:16.027) (total time: 1041ms): Trace[1896351420]: ["GuaranteedUpdate etcd3" audit-id:455f6ddc-6a3d-498a-a08d-d5230dccad5c,key:/leases/kube-system/kube-scheduler,type:*coordination.Lease,resource:leases.coordination.k8s.io 1040ms (20:03:16.027) Trace[1896351420]: ---"Txn call completed" 1039ms (20:03:17.068)] Trace[1896351420]: [1.041047896s] [1.041047896s] END I0628 20:03:17.068291 1 trace.go:236] Trace[1740267922]: "Update" accept:application/json, */*,audit-id:fb3e0027-c655-43db-aa74-f77f2d133351,client:10.0.0.139,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-controller,user-agent:cert-manager-leader-election/v1.12.17 (linux/amd64) cert-manager/37b853ff34a6c093e946c657c189f40413c0f628,verb:PUT (28-Jun-2026 20:03:16.027) (total time: 1040ms): Trace[1740267922]: ["GuaranteedUpdate etcd3" audit-id:fb3e0027-c655-43db-aa74-f77f2d133351,key:/leases/cert-manager/cert-manager-controller,type:*coordination.Lease,resource:leases.coordination.k8s.io 1040ms (20:03:16.027) Trace[1740267922]: ---"Txn call completed" 1039ms (20:03:17.067)] Trace[1740267922]: [1.040833001s] [1.040833001s] END I0628 20:03:17.068803 1 trace.go:236] Trace[1824059991]: "Get" accept:application/json, */*,audit-id:41d2a64d-7a45-4ead-8aac-7b62159ebef5,client:199.204.45.169,protocol:HTTP/2.0,resource:services,scope:resource,url:/api/v1/namespaces/auth-system/services/keycloak-metrics,user-agent:Helm/3.11.2,verb:GET (28-Jun-2026 20:03:16.026) (total time: 1042ms): Trace[1824059991]: ---"About to write a response" 1041ms (20:03:17.067) Trace[1824059991]: [1.042173006s] [1.042173006s] END I0628 20:03:17.069704 1 trace.go:236] Trace[1464270623]: "Update" accept:application/json, */*,audit-id:3228a241-1fdc-46fa-9afc-ea5324594b9a,client:10.0.0.117,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-cainjector-leader-election,user-agent:cainjector/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:03:16.027) (total time: 1041ms): Trace[1464270623]: ["GuaranteedUpdate etcd3" audit-id:3228a241-1fdc-46fa-9afc-ea5324594b9a,key:/leases/cert-manager/cert-manager-cainjector-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 1041ms (20:03:16.028) Trace[1464270623]: ---"Txn call completed" 1040ms (20:03:17.069)] Trace[1464270623]: [1.041748459s] [1.041748459s] END I0628 20:03:17.425706 1 trace.go:236] Trace[1189634643]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.169,type:*v1.Endpoints,resource:apiServerIPInfo (28-Jun-2026 20:03:15.329) (total time: 2096ms): Trace[1189634643]: ---"initial value restored" 696ms (20:03:16.025) Trace[1189634643]: ---"Transaction prepared" 1042ms (20:03:17.067) Trace[1189634643]: ---"Txn call completed" 357ms (20:03:17.425) Trace[1189634643]: [2.096257822s] [2.096257822s] END I0628 20:03:17.429365 1 trace.go:236] Trace[2081099705]: "Get" accept:application/json, */*,audit-id:b93fddd3-7875-41ac-8748-fdb8d9214225,client:10.0.0.187,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/percona-xtradb-pxc-0/log,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format,verb:CONNECT (28-Jun-2026 20:03:16.026) (total time: 1402ms): Trace[2081099705]: ---"About to write a response" 1399ms (20:03:17.425) Trace[2081099705]: [1.402538667s] [1.402538667s] END I0628 20:05:05.465304 1 controller.go:624] quota admission added evaluator for: ingresses.networking.k8s.io I0628 20:05:09.423884 1 handler.go:275] Adding GroupVersion nfd.k8s-sigs.io v1alpha1 to ResourceManager I0628 20:05:09.454309 1 handler.go:275] Adding GroupVersion nfd.k8s-sigs.io v1alpha1 to ResourceManager I0628 20:05:13.664030 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0628 20:05:13.674201 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0628 20:05:13.689790 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0628 20:05:13.699688 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0628 20:05:13.710605 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0628 20:05:13.720535 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0628 20:05:13.730618 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0628 20:05:15.589135 1 controller.go:624] quota admission added evaluator for: nodefeatures.nfd.k8s-sigs.io I0628 20:05:25.915078 1 controller.go:624] quota admission added evaluator for: passwords.secretgen.k14s.io I0628 20:06:05.007964 1 controller.go:624] quota admission added evaluator for: secrettemplates.secretgen.carvel.dev I0628 20:06:19.468654 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0628 20:06:19.653091 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0628 20:06:19.672739 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0628 20:06:19.708436 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0628 20:06:19.987137 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0628 20:06:20.322448 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0628 20:06:20.337221 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0628 20:06:20.391483 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0628 20:06:20.425160 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0628 20:06:20.585740 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0628 20:07:00.963075 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-prometheus" clusterIPs={"IPv4":"10.109.202.114"} I0628 20:07:00.973527 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-operator" clusterIPs={"IPv4":"10.110.112.11"} I0628 20:07:00.989423 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-prometheus-node-exporter" clusterIPs={"IPv4":"10.101.143.126"} I0628 20:07:01.000623 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-grafana" clusterIPs={"IPv4":"10.105.6.93"} I0628 20:07:01.010035 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-alertmanager" clusterIPs={"IPv4":"10.98.182.132"} I0628 20:07:01.019733 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-kube-state-metrics" clusterIPs={"IPv4":"10.106.1.206"} I0628 20:07:01.066841 1 controller.go:624] quota admission added evaluator for: alertmanagers.monitoring.coreos.com I0628 20:07:01.085930 1 controller.go:624] quota admission added evaluator for: prometheusrules.monitoring.coreos.com I0628 20:07:01.129268 1 controller.go:624] quota admission added evaluator for: podmonitors.monitoring.coreos.com I0628 20:07:01.139914 1 controller.go:624] quota admission added evaluator for: servicemonitors.monitoring.coreos.com I0628 20:07:01.156837 1 controller.go:624] quota admission added evaluator for: prometheuses.monitoring.coreos.com W0628 20:07:01.166147 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused W0628 20:07:01.166170 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused W0628 20:07:01.166176 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused E0628 20:07:01.166204 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused E0628 20:07:01.166222 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused W0628 20:07:01.166295 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused E0628 20:07:01.166337 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused W0628 20:07:01.166303 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused W0628 20:07:01.166364 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused E0628 20:07:01.166373 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused W0628 20:07:01.166158 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused E0628 20:07:01.166389 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused E0628 20:07:01.166398 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused W0628 20:07:01.166363 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused E0628 20:07:01.166427 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused W0628 20:07:01.166497 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused E0628 20:07:01.166204 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused W0628 20:07:01.166488 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused E0628 20:07:01.166558 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused E0628 20:07:01.166532 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused W0628 20:07:01.166766 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused E0628 20:07:01.166796 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused W0628 20:07:01.167013 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused E0628 20:07:01.167078 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused W0628 20:07:01.167510 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused W0628 20:07:01.167537 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused E0628 20:07:01.167554 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused W0628 20:07:01.167521 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused E0628 20:07:01.167571 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused W0628 20:07:01.167538 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused E0628 20:07:01.167582 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused E0628 20:07:01.167598 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.110.112.11:443: connect: connection refused I0628 20:07:12.226886 1 alloc.go:330] "allocated clusterIPs" service="monitoring/loki-gateway" clusterIPs={"IPv4":"10.106.38.133"} I0628 20:07:12.237715 1 alloc.go:330] "allocated clusterIPs" service="monitoring/loki" clusterIPs={"IPv4":"10.98.159.182"} I0628 20:07:13.374507 1 trace.go:236] Trace[1124920924]: "Create" accept:application/json, */*,audit-id:2d3e7c27-ceb5-4c59-b077-8071c0267202,client:199.204.45.169,protocol:HTTP/2.0,resource:ciliumidentities,scope:resource,url:/apis/cilium.io/v2/ciliumidentities,user-agent:cilium-agent/1.14.8 cf6e022e 2024-03-13T12:23:35-04:00 go version go1.21.8 linux/amd64,verb:POST (28-Jun-2026 20:07:12.852) (total time: 522ms): Trace[1124920924]: ["Create etcd3" audit-id:2d3e7c27-ceb5-4c59-b077-8071c0267202,key:/cilium.io/ciliumidentities/43563,type:*unstructured.Unstructured,resource:ciliumidentities.cilium.io 520ms (20:07:12.853) Trace[1124920924]: ---"Txn call succeeded" 519ms (20:07:13.373)] Trace[1124920924]: [522.228169ms] [522.228169ms] END I0628 20:07:13.377400 1 trace.go:236] Trace[843476160]: "Get" accept:application/json, */*,audit-id:f08fd999-8873-43ab-885f-44e387be7659,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:07:12.867) (total time: 509ms): Trace[843476160]: ---"About to write a response" 509ms (20:07:13.377) Trace[843476160]: [509.627918ms] [509.627918ms] END I0628 20:07:13.377451 1 trace.go:236] Trace[1036208197]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:0266b073-11d1-48db-9465-df8d824205c1,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:07:12.863) (total time: 514ms): Trace[1036208197]: ---"About to write a response" 513ms (20:07:13.377) Trace[1036208197]: [514.042183ms] [514.042183ms] END I0628 20:07:13.377620 1 trace.go:236] Trace[867673162]: "Get" accept:application/json, */*,audit-id:7c710a0a-c7c5-4738-9038-23620d4f05ff,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:07:12.861) (total time: 515ms): Trace[867673162]: ---"About to write a response" 515ms (20:07:13.377) Trace[867673162]: [515.745405ms] [515.745405ms] END I0628 20:07:13.378039 1 trace.go:236] Trace[1035642837]: "Get" accept:application/json, */*,audit-id:873e5c73-58b4-48ab-847e-5bb9a9afc3a4,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:07:12.864) (total time: 513ms): Trace[1035642837]: ---"About to write a response" 513ms (20:07:13.377) Trace[1035642837]: [513.752297ms] [513.752297ms] END I0628 20:07:13.378210 1 trace.go:236] Trace[1618224887]: "Get" accept:application/json, */*,audit-id:87051166-24ef-45bd-a57c-6c13b28198dd,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/keepalived-qvhr9,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:07:12.859) (total time: 518ms): Trace[1618224887]: ---"About to write a response" 517ms (20:07:13.377) Trace[1618224887]: [518.530337ms] [518.530337ms] END I0628 20:07:13.378721 1 trace.go:236] Trace[2028063545]: "Get" accept:application/json, */*,audit-id:a01892e2-1a78-47be-a68e-dcbcc68f92f9,client:10.0.0.25,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:07:12.866) (total time: 512ms): Trace[2028063545]: ---"About to write a response" 512ms (20:07:13.378) Trace[2028063545]: [512.238657ms] [512.238657ms] END I0628 20:07:16.928521 1 trace.go:236] Trace[2034677181]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:b9bb7e22-827c-46c3-8f02-05e5d7ce7d11,client:10.0.0.170,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:07:16.189) (total time: 738ms): Trace[2034677181]: ["GuaranteedUpdate etcd3" audit-id:b9bb7e22-827c-46c3-8f02-05e5d7ce7d11,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 738ms (20:07:16.189) Trace[2034677181]: ---"Txn call completed" 737ms (20:07:16.928)] Trace[2034677181]: [738.70116ms] [738.70116ms] END I0628 20:07:17.279170 1 trace.go:236] Trace[214872265]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:cd28c449-4953-4b61-8fba-62adc1bd57c5,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:PUT (28-Jun-2026 20:07:16.345) (total time: 934ms): Trace[214872265]: ["GuaranteedUpdate etcd3" audit-id:cd28c449-4953-4b61-8fba-62adc1bd57c5,key:/leases/ingress-nginx/ingress-nginx-leader,type:*coordination.Lease,resource:leases.coordination.k8s.io 933ms (20:07:16.345) Trace[214872265]: ---"Txn call completed" 932ms (20:07:17.278)] Trace[214872265]: [934.070546ms] [934.070546ms] END I0628 20:07:17.279743 1 trace.go:236] Trace[1960726190]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:c3525226-4a1b-4e61-8b94-a31c9f7c5800,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:07:16.600) (total time: 679ms): Trace[1960726190]: ---"About to write a response" 679ms (20:07:17.279) Trace[1960726190]: [679.180391ms] [679.180391ms] END I0628 20:07:17.280227 1 trace.go:236] Trace[149139378]: "Get" accept:application/json, */*,audit-id:e4918399-79f6-47d8-9bef-fb670f7146a4,client:10.0.0.193,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/local-path-storage/pods/helper-pod-create-pvc-75eca3fa-e5b6-4b9c-a97d-072919d550ca,user-agent:local-path-provisioner/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:07:16.394) (total time: 885ms): Trace[149139378]: ---"About to write a response" 884ms (20:07:17.279) Trace[149139378]: [885.188881ms] [885.188881ms] END I0628 20:07:17.284310 1 trace.go:236] Trace[1082602580]: "List" accept:application/json, */*,audit-id:e6dce662-c071-47c6-a9f9-d5f1c2708a16,client:199.204.45.169,protocol:HTTP/2.0,resource:secrets,scope:namespace,url:/api/v1/namespaces/monitoring/secrets,user-agent:Helm/3.11.2,verb:LIST (28-Jun-2026 20:07:16.704) (total time: 579ms): Trace[1082602580]: ["List(recursive=true) etcd3" audit-id:e6dce662-c071-47c6-a9f9-d5f1c2708a16,key:/secrets/monitoring,resourceVersion:,resourceVersionMatch:,limit:0,continue: 579ms (20:07:16.704)] Trace[1082602580]: [579.509964ms] [579.509964ms] END I0628 20:07:18.163354 1 alloc.go:330] "allocated clusterIPs" service="monitoring/goldpinger" clusterIPs={"IPv4":"10.107.152.73"} I0628 20:07:19.851668 1 trace.go:236] Trace[996720115]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:0816a7ad-7b89-4b88-ad4c-2c71cddf273b,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:07:19.310) (total time: 541ms): Trace[996720115]: ---"About to write a response" 541ms (20:07:19.851) Trace[996720115]: [541.249063ms] [541.249063ms] END I0628 20:07:19.862837 1 trace.go:236] Trace[1362464807]: "Delete" accept:application/json, */*,audit-id:83d61311-d4cb-4fca-8197-fd36a5d568b8,client:10.0.0.193,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/local-path-storage/pods/helper-pod-create-pvc-75eca3fa-e5b6-4b9c-a97d-072919d550ca,user-agent:local-path-provisioner/v0.0.0 (linux/amd64) kubernetes/$Format,verb:DELETE (28-Jun-2026 20:07:19.292) (total time: 569ms): Trace[1362464807]: ["GuaranteedUpdate etcd3" audit-id:83d61311-d4cb-4fca-8197-fd36a5d568b8,key:/pods/local-path-storage/helper-pod-create-pvc-75eca3fa-e5b6-4b9c-a97d-072919d550ca,type:*core.Pod,resource:pods 568ms (20:07:19.294) Trace[1362464807]: ---"Txn call completed" 555ms (20:07:19.850)] Trace[1362464807]: [569.795909ms] [569.795909ms] END I0628 20:07:21.085076 1 alloc.go:330] "allocated clusterIPs" service="monitoring/prometheus-smartctl-exporter" clusterIPs={"IPv4":"10.96.95.227"} I0628 20:07:23.161662 1 alloc.go:330] "allocated clusterIPs" service="monitoring/prometheus-pushgateway" clusterIPs={"IPv4":"10.102.79.203"} I0628 20:07:27.547081 1 alloc.go:330] "allocated clusterIPs" service="openstack/memcached" clusterIPs={"IPv4":"10.104.138.243"} I0628 20:07:28.443153 1 alloc.go:330] "allocated clusterIPs" service="openstack/memcached-metrics" clusterIPs={"IPv4":"10.99.92.253"} I0628 20:07:30.975459 1 trace.go:236] Trace[1393447165]: "Update" accept:application/json, */*,audit-id:8396f7d2-ad93-4a66-a573-adef41892939,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Jun-2026 20:07:30.077) (total time: 898ms): Trace[1393447165]: ["GuaranteedUpdate etcd3" audit-id:8396f7d2-ad93-4a66-a573-adef41892939,key:/leases/kube-system/plndr-svcs-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 897ms (20:07:30.077) Trace[1393447165]: ---"Txn call completed" 896ms (20:07:30.975)] Trace[1393447165]: [898.352882ms] [898.352882ms] END I0628 20:07:30.975736 1 trace.go:236] Trace[535706748]: "Update" accept:application/json, */*,audit-id:6ddeb016-5232-47d1-9b4b-b108c288cb82,client:10.0.0.25,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:07:29.964) (total time: 1011ms): Trace[535706748]: ["GuaranteedUpdate etcd3" audit-id:6ddeb016-5232-47d1-9b4b-b108c288cb82,key:/leases/openstack/rabbitmq-cluster-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 1011ms (20:07:29.964) Trace[535706748]: ---"Txn call completed" 1009ms (20:07:30.975)] Trace[535706748]: [1.011485187s] [1.011485187s] END I0628 20:07:30.976055 1 trace.go:236] Trace[1393115600]: "Update" accept:application/json, */*,audit-id:70abc164-c678-46e6-a163-a99133bb4a91,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Jun-2026 20:07:29.967) (total time: 1008ms): Trace[1393115600]: ["GuaranteedUpdate etcd3" audit-id:70abc164-c678-46e6-a163-a99133bb4a91,key:/leases/kube-system/plndr-cp-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 1007ms (20:07:29.968) Trace[1393115600]: ---"Txn call completed" 1006ms (20:07:30.975)] Trace[1393115600]: [1.008213944s] [1.008213944s] END I0628 20:07:30.976057 1 trace.go:236] Trace[238668562]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:43bf3c39-af7d-47c4-96c3-d7413c9967ef,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:07:29.966) (total time: 1009ms): Trace[238668562]: ---"About to write a response" 1009ms (20:07:30.975) Trace[238668562]: [1.009236154s] [1.009236154s] END I0628 20:07:30.976249 1 trace.go:236] Trace[701548297]: "Get" accept:application/json, */*,audit-id:b902781a-d5e7-4e19-a6c9-f11f02cde76b,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:07:29.965) (total time: 1010ms): Trace[701548297]: ---"About to write a response" 1010ms (20:07:30.975) Trace[701548297]: [1.01056708s] [1.01056708s] END I0628 20:07:30.977341 1 trace.go:236] Trace[1142239526]: "List" accept:application/json, */*,audit-id:e3450682-d6f7-4e08-9a50-a31fcee8527a,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:namespace,url:/api/v1/namespaces/openstack/pods,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:LIST (28-Jun-2026 20:07:29.970) (total time: 1006ms): Trace[1142239526]: ["List(recursive=true) etcd3" audit-id:e3450682-d6f7-4e08-9a50-a31fcee8527a,key:/pods/openstack,resourceVersion:,resourceVersionMatch:,limit:0,continue: 1006ms (20:07:29.970)] Trace[1142239526]: [1.00693978s] [1.00693978s] END I0628 20:07:31.069979 1 trace.go:236] Trace[867254945]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:3b38d404-67f9-4f08-bd4a-e10f81543ce2,client:10.0.0.187,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:07:30.420) (total time: 648ms): Trace[867254945]: ["GuaranteedUpdate etcd3" audit-id:3b38d404-67f9-4f08-bd4a-e10f81543ce2,key:/leases/openstack/08db1feb.percona.com,type:*coordination.Lease,resource:leases.coordination.k8s.io 648ms (20:07:30.421) Trace[867254945]: ---"Txn call completed" 647ms (20:07:31.069)] Trace[867254945]: [648.980884ms] [648.980884ms] END I0628 20:07:31.070329 1 trace.go:236] Trace[316307542]: "Update" accept:application/json, */*,audit-id:8c449151-de82-4379-af60-d4e2c8bee320,client:10.0.0.173,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:07:30.425) (total time: 644ms): Trace[316307542]: ["GuaranteedUpdate etcd3" audit-id:8c449151-de82-4379-af60-d4e2c8bee320,key:/leases/openstack/messaging-topology-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 644ms (20:07:30.425) Trace[316307542]: ---"Txn call completed" 643ms (20:07:31.069)] Trace[316307542]: [644.806363ms] [644.806363ms] END I0628 20:07:35.720015 1 trace.go:236] Trace[965086248]: "Update" accept:application/json, */*,audit-id:e1b33eb5-1f77-460d-9258-3e4631580256,client:10.0.0.173,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:07:35.086) (total time: 633ms): Trace[965086248]: ["GuaranteedUpdate etcd3" audit-id:e1b33eb5-1f77-460d-9258-3e4631580256,key:/leases/openstack/messaging-topology-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 632ms (20:07:35.086) Trace[965086248]: ---"Txn call completed" 631ms (20:07:35.719)] Trace[965086248]: [633.273672ms] [633.273672ms] END I0628 20:07:35.720583 1 trace.go:236] Trace[939439177]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:9920125d-ee68-4a9c-9a9d-069d51b1a9f5,client:10.0.0.170,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:07:35.138) (total time: 581ms): Trace[939439177]: ["GuaranteedUpdate etcd3" audit-id:9920125d-ee68-4a9c-9a9d-069d51b1a9f5,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 581ms (20:07:35.138) Trace[939439177]: ---"Txn call completed" 580ms (20:07:35.720)] Trace[939439177]: [581.882519ms] [581.882519ms] END I0628 20:07:35.720669 1 trace.go:236] Trace[1842875245]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:bfdd3c76-21a3-4c65-80cc-5d7021c3b7a4,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:07:35.113) (total time: 607ms): Trace[1842875245]: ---"About to write a response" 607ms (20:07:35.720) Trace[1842875245]: [607.625493ms] [607.625493ms] END I0628 20:07:35.720770 1 trace.go:236] Trace[588587724]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:031b527a-daf4-470c-b39c-e3e8a68ff31f,client:10.0.0.187,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:07:35.079) (total time: 641ms): Trace[588587724]: ["GuaranteedUpdate etcd3" audit-id:031b527a-daf4-470c-b39c-e3e8a68ff31f,key:/leases/openstack/08db1feb.percona.com,type:*coordination.Lease,resource:leases.coordination.k8s.io 641ms (20:07:35.079) Trace[588587724]: ---"Txn call completed" 640ms (20:07:35.720)] Trace[588587724]: [641.229675ms] [641.229675ms] END I0628 20:07:35.721217 1 trace.go:236] Trace[1571666856]: "Get" accept:application/json, */*,audit-id:c0ec7fb7-8059-41bd-8cdd-1734c5200b96,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:07:35.100) (total time: 620ms): Trace[1571666856]: ---"About to write a response" 620ms (20:07:35.721) Trace[1571666856]: [620.431068ms] [620.431068ms] END I0628 20:07:36.710493 1 controller.go:624] quota admission added evaluator for: rabbitmqclusters.rabbitmq.com I0628 20:07:36.871957 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-keystone" clusterIPs={"IPv4":"10.99.72.68"} I0628 20:07:39.570818 1 trace.go:236] Trace[955614196]: "Create" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:33d98654-9645-4f03-b15d-a98a03ca06dd,client:199.204.45.169,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/monitoring/events,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:POST (28-Jun-2026 20:07:38.641) (total time: 929ms): Trace[955614196]: ["Create etcd3" audit-id:33d98654-9645-4f03-b15d-a98a03ca06dd,key:/events/monitoring/prometheus-kube-prometheus-stack-prometheus-0.18bd57858e6aafc1,type:*core.Event,resource:events 928ms (20:07:38.642) Trace[955614196]: ---"Txn call succeeded" 928ms (20:07:39.570)] Trace[955614196]: [929.213753ms] [929.213753ms] END I0628 20:07:39.571114 1 trace.go:236] Trace[1704829974]: "Get" accept:application/json, */*,audit-id:18d1a558-2b53-431d-9e54-68fcae28c58a,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:07:39.014) (total time: 556ms): Trace[1704829974]: ---"About to write a response" 555ms (20:07:39.570) Trace[1704829974]: [556.007386ms] [556.007386ms] END I0628 20:07:39.571468 1 trace.go:236] Trace[43594716]: "Get" accept:application/json, */*,audit-id:cc5601e3-59f3-4e3a-9b60-38f4e9210055,client:10.0.0.193,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/local-path-storage/pods/helper-pod-create-pvc-2a6e538b-d8e9-42a0-bddd-2b46a133ac4d,user-agent:local-path-provisioner/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:07:39.060) (total time: 511ms): Trace[43594716]: ---"About to write a response" 510ms (20:07:39.570) Trace[43594716]: [511.340031ms] [511.340031ms] END I0628 20:07:39.571708 1 trace.go:236] Trace[285935041]: "Get" accept:application/json, */*,audit-id:838b3020-3d15-4647-ba7c-28d870c1e5ae,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:07:39.016) (total time: 554ms): Trace[285935041]: ---"About to write a response" 554ms (20:07:39.571) Trace[285935041]: [554.770961ms] [554.770961ms] END I0628 20:07:39.571818 1 trace.go:236] Trace[188609270]: "Get" accept:application/json, */*,audit-id:04286d5a-f14c-42a9-88ff-946da3cdea4c,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/keepalived-qvhr9,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:07:39.008) (total time: 563ms): Trace[188609270]: ---"About to write a response" 562ms (20:07:39.571) Trace[188609270]: [563.392486ms] [563.392486ms] END I0628 20:07:39.572324 1 trace.go:236] Trace[677527314]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:2b2705a5-ab17-4809-a2df-5d6042121309,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/local-path-storage/pods/helper-pod-create-pvc-2a6e538b-d8e9-42a0-bddd-2b46a133ac4d,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:07:38.650) (total time: 921ms): Trace[677527314]: ---"About to write a response" 921ms (20:07:39.572) Trace[677527314]: [921.888423ms] [921.888423ms] END I0628 20:07:39.572776 1 trace.go:236] Trace[1891591383]: "Get" accept:application/json, */*,audit-id:ec9c3751-5812-4296-bd97-4ef849a93eea,client:10.0.0.25,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:07:39.018) (total time: 554ms): Trace[1891591383]: ---"About to write a response" 554ms (20:07:39.572) Trace[1891591383]: [554.683069ms] [554.683069ms] END I0628 20:07:39.573157 1 trace.go:236] Trace[823881382]: "Update" accept:application/json, */*,audit-id:67c567dc-d7d5-4b8a-8e77-2abf2e1352d0,client:10.0.0.25,protocol:HTTP/2.0,resource:rabbitmqclusters,scope:resource,url:/apis/rabbitmq.com/v1beta1/namespaces/openstack/rabbitmqclusters/rabbitmq-keystone,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Jun-2026 20:07:39.034) (total time: 538ms): Trace[823881382]: ["GuaranteedUpdate etcd3" audit-id:67c567dc-d7d5-4b8a-8e77-2abf2e1352d0,key:/rabbitmq.com/rabbitmqclusters/openstack/rabbitmq-keystone,type:*unstructured.Unstructured,resource:rabbitmqclusters.rabbitmq.com 537ms (20:07:39.036)] Trace[823881382]: [538.234296ms] [538.234296ms] END I0628 20:07:40.127361 1 trace.go:236] Trace[1271219819]: "Create" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:4b23566f-9e85-487d-96e4-c975d3b4619d,client:199.204.45.169,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/monitoring/events,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:POST (28-Jun-2026 20:07:39.572) (total time: 554ms): Trace[1271219819]: ["Create etcd3" audit-id:4b23566f-9e85-487d-96e4-c975d3b4619d,key:/events/monitoring/prometheus-kube-prometheus-stack-prometheus-0.18bd57858e907d21,type:*core.Event,resource:events 553ms (20:07:39.573) Trace[1271219819]: ---"Txn call succeeded" 553ms (20:07:40.126)] Trace[1271219819]: [554.366863ms] [554.366863ms] END I0628 20:07:40.128730 1 trace.go:236] Trace[806752256]: "List" accept:application/json, */*,audit-id:f083ea22-4ddc-45f8-9006-739f060594df,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:namespace,url:/api/v1/namespaces/openstack/pods,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:LIST (28-Jun-2026 20:07:39.574) (total time: 554ms): Trace[806752256]: ["List(recursive=true) etcd3" audit-id:f083ea22-4ddc-45f8-9006-739f060594df,key:/pods/openstack,resourceVersion:,resourceVersionMatch:,limit:0,continue: 554ms (20:07:39.574)] Trace[806752256]: [554.473536ms] [554.473536ms] END I0628 20:07:40.167619 1 trace.go:236] Trace[894300007]: "Update" accept:application/json, */*,audit-id:5984593f-21fe-4b48-82fe-68cd7d39e79a,client:10.0.0.25,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:07:39.574) (total time: 592ms): Trace[894300007]: ["GuaranteedUpdate etcd3" audit-id:5984593f-21fe-4b48-82fe-68cd7d39e79a,key:/leases/openstack/rabbitmq-cluster-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 592ms (20:07:39.575) Trace[894300007]: ---"Txn call completed" 591ms (20:07:40.167)] Trace[894300007]: [592.654286ms] [592.654286ms] END I0628 20:07:40.167983 1 trace.go:236] Trace[490001567]: "Update" accept:application/json, */*,audit-id:ea4fcecb-eff1-492b-9d1e-3847b3f51850,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Jun-2026 20:07:39.577) (total time: 589ms): Trace[490001567]: ["GuaranteedUpdate etcd3" audit-id:ea4fcecb-eff1-492b-9d1e-3847b3f51850,key:/leases/kube-system/plndr-svcs-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 589ms (20:07:39.578) Trace[490001567]: ---"Txn call completed" 588ms (20:07:40.167)] Trace[490001567]: [589.966766ms] [589.966766ms] END I0628 20:07:40.168082 1 trace.go:236] Trace[1076178308]: "Update" accept:application/json, */*,audit-id:c52d71d6-dd34-4ab7-adbf-737bf420ed19,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Jun-2026 20:07:39.576) (total time: 591ms): Trace[1076178308]: ["GuaranteedUpdate etcd3" audit-id:c52d71d6-dd34-4ab7-adbf-737bf420ed19,key:/leases/kube-system/plndr-cp-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 591ms (20:07:39.576) Trace[1076178308]: ---"Txn call completed" 590ms (20:07:40.167)] Trace[1076178308]: [591.956353ms] [591.956353ms] END I0628 20:07:40.168202 1 trace.go:236] Trace[335742004]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:5be67c28-2340-46a1-a92c-3d6174fda161,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/local-path-storage/pods/helper-pod-create-pvc-2a6e538b-d8e9-42a0-bddd-2b46a133ac4d/status,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (28-Jun-2026 20:07:39.574) (total time: 593ms): Trace[335742004]: ["GuaranteedUpdate etcd3" audit-id:5be67c28-2340-46a1-a92c-3d6174fda161,key:/pods/local-path-storage/helper-pod-create-pvc-2a6e538b-d8e9-42a0-bddd-2b46a133ac4d,type:*core.Pod,resource:pods 593ms (20:07:39.574) Trace[335742004]: ---"Txn call completed" 589ms (20:07:40.167)] Trace[335742004]: ---"Object stored in database" 589ms (20:07:40.168) Trace[335742004]: [593.40193ms] [593.40193ms] END I0628 20:07:40.169106 1 trace.go:236] Trace[1910358099]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:22e0e119-b9df-47b9-86c3-1463aad61fbd,client:10.0.0.25,protocol:HTTP/2.0,resource:secrets,scope:resource,url:/api/v1/namespaces/openstack/secrets/rabbitmq-keystone-default-user,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:07:39.575) (total time: 592ms): Trace[1910358099]: ---"About to write a response" 592ms (20:07:40.168) Trace[1910358099]: [592.556984ms] [592.556984ms] END I0628 20:07:54.486092 1 trace.go:236] Trace[845934652]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:e198907c-0e77-42c0-aaf1-dd554a10c2e2,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/rabbitmq-keystone-server-0/status,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (28-Jun-2026 20:07:53.844) (total time: 641ms): Trace[845934652]: ["GuaranteedUpdate etcd3" audit-id:e198907c-0e77-42c0-aaf1-dd554a10c2e2,key:/pods/openstack/rabbitmq-keystone-server-0,type:*core.Pod,resource:pods 641ms (20:07:53.844) Trace[845934652]: ---"Txn call completed" 633ms (20:07:54.485)] Trace[845934652]: ---"Object stored in database" 634ms (20:07:54.485) Trace[845934652]: [641.882991ms] [641.882991ms] END I0628 20:07:54.534443 1 trace.go:236] Trace[2050046318]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:dea7033b-e625-4321-bb18-9b042e55d31f,client:199.204.45.169,protocol:HTTP/2.0,resource:persistentvolumeclaims,scope:resource,url:/api/v1/namespaces/openstack/persistentvolumeclaims/persistence-rabbitmq-keystone-server-0,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:07:53.911) (total time: 622ms): Trace[2050046318]: ---"About to write a response" 622ms (20:07:54.534) Trace[2050046318]: [622.947221ms] [622.947221ms] END I0628 20:08:28.317590 1 alloc.go:330] "allocated clusterIPs" service="openstack/keystone-api" clusterIPs={"IPv4":"10.107.182.147"} I0628 20:08:28.334149 1 controller.go:624] quota admission added evaluator for: cronjobs.batch I0628 20:11:31.122577 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-barbican" clusterIPs={"IPv4":"10.97.250.100"} I0628 20:11:40.790121 1 metrics.go:285] "Failed to get storage metrics" storage_cluster_id="etcd-0" err="context deadline exceeded" E0628 20:11:43.878946 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0628 20:11:43.879105 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:11:43.880671 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:11:43.880772 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:11:43.881911 1 trace.go:236] Trace[1573586101]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:3d39a00f-ecd9-4b0e-9516-c4ed5a4eadf8,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:11:38.878) (total time: 5003ms): Trace[1573586101]: [5.003514489s] [5.003514489s] END E0628 20:11:43.882137 1 timeout.go:142] post-timeout activity - time-elapsed: 2.910225ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: E0628 20:11:44.001533 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0628 20:11:44.001654 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:11:44.002897 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:11:44.003075 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:11:44.004080 1 trace.go:236] Trace[1036702755]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:5fc1d504-f60c-4912-82b9-00cb9f61783d,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:11:39.001) (total time: 5002ms): Trace[1036702755]: [5.002402867s] [5.002402867s] END E0628 20:11:44.004252 1 timeout.go:142] post-timeout activity - time-elapsed: 2.495347ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: E0628 20:11:44.547959 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:11:44.548079 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 12.94µs, panicked: false, err: context canceled, panic-reason: E0628 20:11:44.548205 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:11:44.549675 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:11:44.549763 1 trace.go:236] Trace[1460983636]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:36c79fa4-3d72-4492-b258-c922cb28c068,client:10.0.0.187,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:11:39.548) (total time: 5001ms): Trace[1460983636]: ["GuaranteedUpdate etcd3" audit-id:36c79fa4-3d72-4492-b258-c922cb28c068,key:/leases/openstack/08db1feb.percona.com,type:*coordination.Lease,resource:leases.coordination.k8s.io 5000ms (20:11:39.548) Trace[1460983636]: ---"Txn call failed" err:context canceled 4998ms (20:11:44.547)] Trace[1460983636]: [5.001145003s] [5.001145003s] END E0628 20:11:44.549973 1 timeout.go:142] post-timeout activity - time-elapsed: 1.944247ms, PUT "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com" result: E0628 20:11:44.621950 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:11:44.621987 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:11:44.622037 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 20.231µs, panicked: false, err: context deadline exceeded, panic-reason: E0628 20:11:44.623252 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:11:44.623361 1 trace.go:236] Trace[2083184383]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:7331da54-72f5-49f0-9c8f-776284963a78,client:10.0.0.170,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:11:39.622) (total time: 5000ms): Trace[2083184383]: ["GuaranteedUpdate etcd3" audit-id:7331da54-72f5-49f0-9c8f-776284963a78,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 5000ms (20:11:39.622) Trace[2083184383]: ---"Txn call failed" err:context deadline exceeded 4998ms (20:11:44.621)] Trace[2083184383]: [5.000843166s] [5.000843166s] END E0628 20:11:44.623525 1 timeout.go:142] post-timeout activity - time-elapsed: 1.56846ms, PUT "/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io" result: E0628 20:11:45.551540 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0628 20:11:45.551645 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:11:45.552810 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:11:45.552962 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:11:45.553953 1 trace.go:236] Trace[1555555062]: "Get" accept:application/json, */*,audit-id:89c0dfe8-28af-40a6-bde7-3e75a5d37ed4,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:11:40.550) (total time: 5003ms): Trace[1555555062]: [5.003209283s] [5.003209283s] END E0628 20:11:45.554106 1 timeout.go:142] post-timeout activity - time-elapsed: 2.61285ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: E0628 20:11:45.784869 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:11:45.785057 1 trace.go:236] Trace[733227569]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:68ea9447-cb5a-42fe-b45c-18e5129e7e0a,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/rabbitmq-barbican-server-0/status,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (28-Jun-2026 20:11:38.777) (total time: 7007ms): Trace[733227569]: ["GuaranteedUpdate etcd3" audit-id:68ea9447-cb5a-42fe-b45c-18e5129e7e0a,key:/pods/openstack/rabbitmq-barbican-server-0,type:*core.Pod,resource:pods 7007ms (20:11:38.777) Trace[733227569]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (20:11:45.784)] Trace[733227569]: [7.007531986s] [7.007531986s] END E0628 20:11:45.875005 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:11:45.875176 1 trace.go:236] Trace[904529328]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:cd9ddd06-7bc5-405c-895b-365a8fde8900,client:199.204.45.169,protocol:HTTP/2.0,resource:persistentvolumeclaims,scope:resource,url:/api/v1/namespaces/openstack/persistentvolumeclaims/persistence-rabbitmq-barbican-server-0,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:11:38.872) (total time: 7002ms): Trace[904529328]: [7.002332336s] [7.002332336s] END E0628 20:11:46.747006 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:11:46.747263 1 trace.go:236] Trace[1723839503]: "Update" accept:application/json, */*,audit-id:de959313-1bef-4c2f-aef1-7afa02e02f46,client:10.0.0.173,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:11:39.743) (total time: 7003ms): Trace[1723839503]: ["GuaranteedUpdate etcd3" audit-id:de959313-1bef-4c2f-aef1-7afa02e02f46,key:/leases/openstack/messaging-topology-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (20:11:39.743) Trace[1723839503]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (20:11:46.746)] Trace[1723839503]: [7.003662561s] [7.003662561s] END E0628 20:11:47.446056 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:11:47.446273 1 trace.go:236] Trace[2074114831]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:b6ae88a5-1d9c-4c74-a131-aa935c459fe5,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:PUT (28-Jun-2026 20:11:40.441) (total time: 7004ms): Trace[2074114831]: ["GuaranteedUpdate etcd3" audit-id:b6ae88a5-1d9c-4c74-a131-aa935c459fe5,key:/leases/ingress-nginx/ingress-nginx-leader,type:*coordination.Lease,resource:leases.coordination.k8s.io 7004ms (20:11:40.441) Trace[2074114831]: ---"Txn call failed" err:etcdserver: request timed out 7003ms (20:11:47.445)] Trace[2074114831]: [7.004872245s] [7.004872245s] END E0628 20:11:48.397464 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:11:48.397728 1 trace.go:236] Trace[449325268]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:17257929-fbe7-4fd9-8557-2c8ccc5b23a2,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Jun-2026 20:11:41.394) (total time: 7002ms): Trace[449325268]: ["GuaranteedUpdate etcd3" audit-id:17257929-fbe7-4fd9-8557-2c8ccc5b23a2,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 7002ms (20:11:41.394) Trace[449325268]: ---"Txn call failed" err:etcdserver: request timed out 7001ms (20:11:48.397)] Trace[449325268]: [7.002877108s] [7.002877108s] END E0628 20:11:48.399316 1 controller.go:193] "Failed to update lease" err="etcdserver: request timed out" E0628 20:11:48.878347 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0628 20:11:48.878455 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:11:48.879570 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:11:48.879607 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:11:48.880874 1 trace.go:236] Trace[803053815]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:e444d984-4a09-425d-8206-d87c8b5e8399,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:11:45.880) (total time: 3000ms): Trace[803053815]: [3.000378368s] [3.000378368s] END E0628 20:11:48.881288 1 timeout.go:142] post-timeout activity - time-elapsed: 2.963367ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: E0628 20:11:49.001111 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0628 20:11:49.001247 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:11:49.002430 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:11:49.002488 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:11:49.003677 1 trace.go:236] Trace[1535109216]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:60f579d5-27a3-49c0-9cc6-f0507473ffe4,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:11:46.005) (total time: 2998ms): Trace[1535109216]: [2.998443682s] [2.998443682s] END E0628 20:11:49.003843 1 timeout.go:142] post-timeout activity - time-elapsed: 2.716162ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: E0628 20:11:49.547793 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0628 20:11:49.547971 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:11:49.549975 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:11:49.550077 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:11:49.551167 1 trace.go:236] Trace[1478186531]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:8a1b1e3d-4a75-463f-8907-0bd9d900b835,client:10.0.0.187,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:11:44.549) (total time: 5001ms): Trace[1478186531]: [5.001935959s] [5.001935959s] END E0628 20:11:49.551380 1 timeout.go:142] post-timeout activity - time-elapsed: 3.612219ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com" result: E0628 20:11:49.575747 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled I0628 20:11:49.575893 1 trace.go:236] Trace[770372943]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:81fdac26-4d32-4028-b739-d4f08a4513c2,client:10.0.0.187,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/percona-xtradb-pxc-0,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:11:42.918) (total time: 6656ms): Trace[770372943]: [6.656996961s] [6.656996961s] END E0628 20:11:49.575996 1 wrap.go:54] timeout or abort while handling: method=GET URI="/api/v1/namespaces/openstack/pods/percona-xtradb-pxc-0" audit-ID="81fdac26-4d32-4028-b739-d4f08a4513c2" E0628 20:11:49.576107 1 timeout.go:142] post-timeout activity - time-elapsed: 8.61µs, GET "/api/v1/namespaces/openstack/pods/percona-xtradb-pxc-0" result: E0628 20:11:49.622245 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0628 20:11:49.622400 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:11:49.623453 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:11:49.623532 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:11:49.624583 1 trace.go:236] Trace[1636263227]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:bfa11b8d-7bbf-46e8-a32c-3ef9e2d4092b,client:10.0.0.170,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:11:44.625) (total time: 4999ms): Trace[1636263227]: [4.999176306s] [4.999176306s] END E0628 20:11:49.624801 1 timeout.go:142] post-timeout activity - time-elapsed: 2.538207ms, GET "/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io" result: E0628 20:11:49.743077 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0628 20:11:49.743310 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:11:49.744236 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:11:49.745327 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:11:49.745414 1 trace.go:236] Trace[1561056219]: "Get" accept:application/json, */*,audit-id:6d3bda93-a885-4b84-bed0-489e5787d032,client:10.0.0.173,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:11:46.749) (total time: 2995ms): Trace[1561056219]: [2.995750989s] [2.995750989s] END E0628 20:11:49.745582 1 timeout.go:142] post-timeout activity - time-elapsed: 2.482727ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election" result: E0628 20:11:50.449982 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0628 20:11:50.450228 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:11:50.451189 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:11:50.451356 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:11:50.452539 1 trace.go:236] Trace[1566259838]: "Get" accept:application/json, */*,audit-id:46ca14bb-af5c-4786-a574-31d6356030c8,client:10.0.0.25,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:11:40.449) (total time: 10002ms): Trace[1566259838]: [10.002828226s] [10.002828226s] END E0628 20:11:50.452740 1 timeout.go:142] post-timeout activity - time-elapsed: 2.998777ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election" result: E0628 20:11:50.456454 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:11:50.456489 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:11:50.456503 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 13.75µs, panicked: false, err: context canceled, panic-reason: E0628 20:11:50.458146 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:11:50.458266 1 trace.go:236] Trace[1829933912]: "Update" accept:application/json, */*,audit-id:74e5d103-287e-4500-98e2-65b835073a82,client:10.0.0.25,protocol:HTTP/2.0,resource:rabbitmqclusters,scope:resource,url:/apis/rabbitmq.com/v1beta1/namespaces/openstack/rabbitmqclusters/rabbitmq-barbican,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Jun-2026 20:11:48.498) (total time: 1959ms): Trace[1829933912]: ["GuaranteedUpdate etcd3" audit-id:74e5d103-287e-4500-98e2-65b835073a82,key:/rabbitmq.com/rabbitmqclusters/openstack/rabbitmq-barbican,type:*unstructured.Unstructured,resource:rabbitmqclusters.rabbitmq.com 1958ms (20:11:48.499)] Trace[1829933912]: [1.959776045s] [1.959776045s] END E0628 20:11:50.458542 1 timeout.go:142] post-timeout activity - time-elapsed: 2.021489ms, PUT "/apis/rabbitmq.com/v1beta1/namespaces/openstack/rabbitmqclusters/rabbitmq-barbican" result: E0628 20:11:50.550537 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0628 20:11:50.550805 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:11:50.551647 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:11:50.551938 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout E0628 20:11:50.552637 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled I0628 20:11:50.552823 1 trace.go:236] Trace[54370002]: "Get" accept:application/json, */*,audit-id:c87b69c6-7f15-418a-9761-1930f3c3b9c7,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:11:47.553) (total time: 2998ms): Trace[54370002]: [2.998812529s] [2.998812529s] END E0628 20:11:50.553013 1 timeout.go:142] post-timeout activity - time-elapsed: 2.557649ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: E0628 20:11:50.553144 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:11:50.554090 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:11:50.554283 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:11:50.555306 1 trace.go:236] Trace[1830912791]: "Get" accept:application/json, */*,audit-id:ba7b63a0-e34f-4d8d-8522-d475db6cfecc,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:11:40.552) (total time: 10002ms): Trace[1830912791]: [10.002330788s] [10.002330788s] END E0628 20:11:50.555479 1 timeout.go:142] post-timeout activity - time-elapsed: 2.942116ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock" result: E0628 20:11:50.775979 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0628 20:11:50.776171 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:11:50.777564 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:11:50.777669 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:11:50.778701 1 trace.go:236] Trace[182523731]: "Get" accept:application/json, */*,audit-id:410bf5f9-dcad-4872-b5a5-7005ae321d69,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:11:40.776) (total time: 10002ms): Trace[182523731]: [10.002345507s] [10.002345507s] END E0628 20:11:50.778900 1 timeout.go:142] post-timeout activity - time-elapsed: 2.893575ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock" result: E0628 20:11:52.876999 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:11:52.877124 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:11:52.877222 1 trace.go:236] Trace[1497433777]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.169,type:*v1.Endpoints,resource:apiServerIPInfo (28-Jun-2026 20:11:45.360) (total time: 7516ms): Trace[1497433777]: [7.516652449s] [7.516652449s] END E0628 20:11:52.877271 1 controller.go:159] unable to sync kubernetes service: etcdserver: request timed out I0628 20:11:52.877274 1 trace.go:236] Trace[1964786295]: "Get" accept:application/json, */*,audit-id:bc6f2bba-fbbe-4923-a404-03d8b4fc4952,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/keepalived-qvhr9,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:11:40.619) (total time: 12258ms): Trace[1964786295]: [12.258132089s] [12.258132089s] END E0628 20:11:52.877366 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:11:52.878750 1 trace.go:236] Trace[1074929568]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:31e03949-5cf7-4cf6-be6f-0de4d2923ee5,client:199.204.45.169,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/kube-system/events/kube-apiserver-instance.18bd572870f099b7,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (28-Jun-2026 20:11:43.668) (total time: 9210ms): Trace[1074929568]: ["GuaranteedUpdate etcd3" audit-id:31e03949-5cf7-4cf6-be6f-0de4d2923ee5,key:/events/kube-system/kube-apiserver-instance.18bd572870f099b7,type:*core.Event,resource:events 9210ms (20:11:43.668)] Trace[1074929568]: [9.210254926s] [9.210254926s] END I0628 20:11:52.878863 1 trace.go:236] Trace[2122067270]: "Get" accept:application/json,audit-id:87dc578f-d7fe-474d-895f-f1e16d764df7,client:199.204.45.169,protocol:HTTP/1.1,resource:rabbitmqclusters,scope:resource,url:/apis/rabbitmq.com/v1beta1/namespaces/openstack/rabbitmqclusters/rabbitmq-barbican,user-agent:OpenAPI-Generator/36.0.2/python,verb:GET (28-Jun-2026 20:11:41.021) (total time: 11857ms): Trace[2122067270]: [11.857722565s] [11.857722565s] END E0628 20:11:53.994944 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:11:53.995206 1 trace.go:236] Trace[715784555]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:bfba8a60-591d-49d1-a5c8-1242a57800f3,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Jun-2026 20:11:46.991) (total time: 7003ms): Trace[715784555]: ["GuaranteedUpdate etcd3" audit-id:bfba8a60-591d-49d1-a5c8-1242a57800f3,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (20:11:46.991) Trace[715784555]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (20:11:53.994)] Trace[715784555]: [7.003581001s] [7.003581001s] END E0628 20:11:54.002765 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:11:54.003021 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:11:54.002791 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 18.861µs, panicked: false, err: context deadline exceeded, panic-reason: E0628 20:11:54.006698 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:11:54.006821 1 trace.go:236] Trace[425549024]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:c827cdec-89cc-40f6-a720-851fd11b644f,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (28-Jun-2026 20:11:49.002) (total time: 5004ms): Trace[425549024]: ["GuaranteedUpdate etcd3" audit-id:c827cdec-89cc-40f6-a720-851fd11b644f,key:/leases/kube-system/kube-scheduler,type:*coordination.Lease,resource:leases.coordination.k8s.io 5004ms (20:11:49.002) Trace[425549024]: ---"Txn call failed" err:context deadline exceeded 4998ms (20:11:54.002)] Trace[425549024]: [5.004395094s] [5.004395094s] END E0628 20:11:54.007047 1 timeout.go:142] post-timeout activity - time-elapsed: 4.606347ms, PUT "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: E0628 20:11:55.403196 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:11:55.403425 1 trace.go:236] Trace[1055854439]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:f5c681ab-e317-469c-8a0c-bb98491786ee,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Jun-2026 20:11:48.400) (total time: 7003ms): Trace[1055854439]: ["GuaranteedUpdate etcd3" audit-id:f5c681ab-e317-469c-8a0c-bb98491786ee,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 7002ms (20:11:48.400) Trace[1055854439]: ---"Txn call failed" err:etcdserver: request timed out 7001ms (20:11:55.403)] Trace[1055854439]: [7.003064551s] [7.003064551s] END E0628 20:11:55.404157 1 controller.go:193] "Failed to update lease" err="etcdserver: request timed out" E0628 20:11:55.439963 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0628 20:11:55.440102 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:11:55.441143 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:11:55.441226 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:11:55.442317 1 trace.go:236] Trace[892594133]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:e8121c8d-e57f-4cdd-a14c-f3d4f969cbe0,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:GET (28-Jun-2026 20:11:47.449) (total time: 7992ms): Trace[892594133]: [7.992411507s] [7.992411507s] END E0628 20:11:55.442511 1 timeout.go:142] post-timeout activity - time-elapsed: 2.523607ms, GET "/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader" result: E0628 20:11:55.552896 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:11:55.552954 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:11:55.553064 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 13.18µs, panicked: false, err: context deadline exceeded, panic-reason: E0628 20:11:55.554070 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:11:55.554249 1 trace.go:236] Trace[622222617]: "Update" accept:application/json, */*,audit-id:f2297dfb-21a1-4b90-b2db-56af1b444504,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:11:50.552) (total time: 5001ms): Trace[622222617]: ["GuaranteedUpdate etcd3" audit-id:f2297dfb-21a1-4b90-b2db-56af1b444504,key:/leases/kube-system/cilium-operator-resource-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 5001ms (20:11:50.553) Trace[622222617]: ---"Txn call failed" err:context deadline exceeded 4998ms (20:11:55.552)] Trace[622222617]: [5.001348587s] [5.001348587s] END E0628 20:11:55.554434 1 timeout.go:142] post-timeout activity - time-elapsed: 1.671711ms, PUT "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: E0628 20:11:56.629428 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:11:56.629673 1 trace.go:236] Trace[449540908]: "Create" accept:application/json, */*,audit-id:0a35bbc3-c8e5-4cb2-8ba3-117c23b16bbc,client:10.0.0.170,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/envoy-gateway-system/events,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format,verb:POST (28-Jun-2026 20:11:49.625) (total time: 7004ms): Trace[449540908]: ["Create etcd3" audit-id:0a35bbc3-c8e5-4cb2-8ba3-117c23b16bbc,key:/events/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io.18bd57bffe1ec65f,type:*core.Event,resource:events 7002ms (20:11:49.626) Trace[449540908]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (20:11:56.629)] Trace[449540908]: [7.004039958s] [7.004039958s] END E0628 20:11:57.557391 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:11:57.557566 1 trace.go:236] Trace[862516989]: "Update" accept:application/json, */*,audit-id:cd6d59de-dff6-4471-a502-d68054cee18d,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Jun-2026 20:11:50.553) (total time: 7003ms): Trace[862516989]: ["GuaranteedUpdate etcd3" audit-id:cd6d59de-dff6-4471-a502-d68054cee18d,key:/leases/kube-system/plndr-cp-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (20:11:50.554) Trace[862516989]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (20:11:57.557)] Trace[862516989]: [7.00362024s] [7.00362024s] END E0628 20:11:57.566080 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:11:57.566126 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:11:57.566136 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 58.121µs, panicked: false, err: context canceled, panic-reason: E0628 20:11:57.567377 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:11:57.567505 1 trace.go:236] Trace[1846846383]: "Update" accept:application/json, */*,audit-id:109819ea-b298-4924-9e61-609d4208b832,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Jun-2026 20:11:50.777) (total time: 6790ms): Trace[1846846383]: ["GuaranteedUpdate etcd3" audit-id:109819ea-b298-4924-9e61-609d4208b832,key:/leases/kube-system/plndr-svcs-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 6789ms (20:11:50.777) Trace[1846846383]: ---"Txn call failed" err:context canceled 6787ms (20:11:57.566)] Trace[1846846383]: [6.790082364s] [6.790082364s] END E0628 20:11:57.567816 1 timeout.go:142] post-timeout activity - time-elapsed: 1.859205ms, PUT "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock" result: E0628 20:11:59.877068 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:11:59.877117 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:11:59.877251 1 trace.go:236] Trace[649732448]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:4ab12e77-4fce-4c9a-a6ed-6c797551e82e,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/rabbitmq-barbican-server-0,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:11:47.205) (total time: 12671ms): Trace[649732448]: [12.671303355s] [12.671303355s] END E0628 20:11:59.877799 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:11:59.877894 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:11:59.878717 1 trace.go:236] Trace[1327099266]: "Get" accept:application/json, */*,audit-id:0ef9cb4b-521f-4c4b-a1ad-9b5f4ef3361d,client:10.0.0.117,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-cainjector-leader-election,user-agent:cainjector/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:11:47.497) (total time: 12380ms): Trace[1327099266]: [12.380641622s] [12.380641622s] END I0628 20:11:59.878717 1 trace.go:236] Trace[1833335333]: "Get" accept:application/json, */*,audit-id:0bd590c2-0229-40fc-b71e-4c6b34b17dd8,client:10.0.0.139,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-controller,user-agent:cert-manager-leader-election/v1.12.17 (linux/amd64) cert-manager/37b853ff34a6c093e946c657c189f40413c0f628,verb:GET (28-Jun-2026 20:11:48.150) (total time: 11728ms): Trace[1833335333]: [11.728101748s] [11.728101748s] END I0628 20:11:59.879006 1 trace.go:236] Trace[1842709628]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:ae138897-fef8-48d8-9306-caf6a099b890,client:199.204.45.169,protocol:HTTP/2.0,resource:persistentvolumeclaims,scope:resource,url:/api/v1/namespaces/openstack/persistentvolumeclaims/datadir-percona-xtradb-pxc-0,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:11:45.877) (total time: 14001ms): Trace[1842709628]: [14.001285299s] [14.001285299s] END E0628 20:12:01.003748 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:01.003933 1 trace.go:236] Trace[999715758]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:411d7d69-8d08-4106-b275-d4deac2b1f52,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Jun-2026 20:11:53.998) (total time: 7005ms): Trace[999715758]: ["GuaranteedUpdate etcd3" audit-id:411d7d69-8d08-4106-b275-d4deac2b1f52,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 7004ms (20:11:53.998) Trace[999715758]: ---"Txn call failed" err:etcdserver: request timed out 7003ms (20:12:01.003)] Trace[999715758]: [7.005206792s] [7.005206792s] END E0628 20:12:02.409234 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:02.409518 1 trace.go:236] Trace[416877281]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:07badc62-c362-46ee-af0f-b3fd849ff1fd,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Jun-2026 20:11:55.405) (total time: 7004ms): Trace[416877281]: ["GuaranteedUpdate etcd3" audit-id:07badc62-c362-46ee-af0f-b3fd849ff1fd,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 7004ms (20:11:55.405) Trace[416877281]: ---"Txn call failed" err:etcdserver: request timed out 7003ms (20:12:02.409)] Trace[416877281]: [7.004234863s] [7.004234863s] END E0628 20:12:02.410307 1 controller.go:193] "Failed to update lease" err="etcdserver: request timed out" E0628 20:12:02.448262 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:02.448468 1 trace.go:236] Trace[918778933]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:04687d3e-25db-40ff-b6df-96d4262a151d,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:PUT (28-Jun-2026 20:11:55.443) (total time: 7005ms): Trace[918778933]: ["GuaranteedUpdate etcd3" audit-id:04687d3e-25db-40ff-b6df-96d4262a151d,key:/leases/ingress-nginx/ingress-nginx-leader,type:*coordination.Lease,resource:leases.coordination.k8s.io 7005ms (20:11:55.443) Trace[918778933]: ---"Txn call failed" err:etcdserver: request timed out 7003ms (20:12:02.448)] Trace[918778933]: [7.005207891s] [7.005207891s] END E0628 20:12:06.877573 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:12:06.878012 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:06.878075 1 trace.go:236] Trace[149923499]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.169,type:*v1.Endpoints,resource:apiServerIPInfo (28-Jun-2026 20:11:55.361) (total time: 11516ms): Trace[149923499]: [11.516697032s] [11.516697032s] END E0628 20:12:06.878121 1 controller.go:159] unable to sync kubernetes service: etcdserver: request timed out E0628 20:12:06.878242 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:06.878423 1 trace.go:236] Trace[1198089363]: "Get" accept:application/json, */*,audit-id:3d39ae5f-887a-4806-bca5-40556452197d,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/keepalived-qvhr9,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:11:54.880) (total time: 11997ms): Trace[1198089363]: [11.997497616s] [11.997497616s] END I0628 20:12:06.878969 1 trace.go:236] Trace[1382861332]: "List" accept:application/json,audit-id:df3b9488-9227-436c-9fa7-9efefe0401a4,client:199.204.45.169,protocol:HTTP/2.0,resource:namespaces,scope:cluster,url:/api/v1/namespaces,user-agent:kubectl/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:LIST (28-Jun-2026 20:11:57.512) (total time: 9366ms): Trace[1382861332]: ["List(recursive=true) etcd3" audit-id:df3b9488-9227-436c-9fa7-9efefe0401a4,key:/namespaces,resourceVersion:,resourceVersionMatch:,limit:500,continue: 9366ms (20:11:57.512)] Trace[1382861332]: [9.366315958s] [9.366315958s] END I0628 20:12:06.879400 1 trace.go:236] Trace[602758780]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:1e5452fb-a67f-4d99-9983-2a711a321edc,client:199.204.45.169,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/kube-system/events/kube-apiserver-instance.18bd572870f099b7,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (28-Jun-2026 20:11:52.881) (total time: 13998ms): Trace[602758780]: ["GuaranteedUpdate etcd3" audit-id:1e5452fb-a67f-4d99-9983-2a711a321edc,key:/events/kube-system/kube-apiserver-instance.18bd572870f099b7,type:*core.Event,resource:events 13997ms (20:11:52.881)] Trace[602758780]: [13.998062687s] [13.998062687s] END I0628 20:12:07.457311 1 trace.go:236] Trace[540161862]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:8592e315-5715-4df5-b4c8-1d4f409f37ca,client:199.204.45.169,protocol:HTTP/2.0,resource:persistentvolumeclaims,scope:resource,url:/api/v1/namespaces/openstack/persistentvolumeclaims/persistence-rabbitmq-keystone-server-0,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:11:59.983) (total time: 7474ms): Trace[540161862]: ---"About to write a response" 7473ms (20:12:07.457) Trace[540161862]: [7.47411905s] [7.47411905s] END I0628 20:12:07.457547 1 trace.go:236] Trace[2018369366]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:f2960456-3420-4d2d-a127-62708a75d691,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:GET (28-Jun-2026 20:12:02.450) (total time: 5006ms): Trace[2018369366]: ---"About to write a response" 5006ms (20:12:07.457) Trace[2018369366]: [5.006930573s] [5.006930573s] END I0628 20:12:07.457647 1 trace.go:236] Trace[312605916]: "Create" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:bf53e51d-7bb6-4042-a5f5-f4cd4a834349,client:199.204.45.169,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/kube-system/events,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:POST (28-Jun-2026 20:12:06.882) (total time: 575ms): Trace[312605916]: ["Create etcd3" audit-id:bf53e51d-7bb6-4042-a5f5-f4cd4a834349,key:/events/kube-system/kube-apiserver-instance.18bd57bfb0199995,type:*core.Event,resource:events 574ms (20:12:06.882) Trace[312605916]: ---"Txn call succeeded" 574ms (20:12:07.457)] Trace[312605916]: [575.454597ms] [575.454597ms] END I0628 20:12:07.458223 1 trace.go:236] Trace[1452242636]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:02b902d5-e990-48af-92b4-eada754e40b5,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/rabbitmq-barbican-server-0,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:11:59.879) (total time: 7578ms): Trace[1452242636]: ---"About to write a response" 7577ms (20:12:07.457) Trace[1452242636]: [7.578429785s] [7.578429785s] END I0628 20:12:07.458829 1 trace.go:236] Trace[39164743]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:91178cda-20c7-496d-b89a-2780fe8bc19d,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Jun-2026 20:12:02.411) (total time: 5047ms): Trace[39164743]: ["GuaranteedUpdate etcd3" audit-id:91178cda-20c7-496d-b89a-2780fe8bc19d,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 5047ms (20:12:02.411) Trace[39164743]: ---"Txn call completed" 5043ms (20:12:07.457)] Trace[39164743]: [5.047284091s] [5.047284091s] END E0628 20:12:07.461818 1 controller.go:193] "Failed to update lease" err="Operation cannot be fulfilled on leases.coordination.k8s.io \"apiserver-f3tcohoifagyom4bbi4wgeu7te\": the object has been modified; please apply your changes to the latest version and try again" I0628 20:12:07.465678 1 trace.go:236] Trace[1721058294]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:7aa28764-5c94-4450-830c-720ea1209888,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Jun-2026 20:12:01.006) (total time: 6459ms): Trace[1721058294]: ["GuaranteedUpdate etcd3" audit-id:7aa28764-5c94-4450-830c-720ea1209888,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 6459ms (20:12:01.006) Trace[1721058294]: ---"Txn call completed" 6448ms (20:12:07.456)] Trace[1721058294]: [6.45939545s] [6.45939545s] END I0628 20:12:07.507132 1 trace.go:236] Trace[1250878695]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.169,type:*v1.Endpoints,resource:apiServerIPInfo (28-Jun-2026 20:12:06.879) (total time: 627ms): Trace[1250878695]: ---"initial value restored" 579ms (20:12:07.458) Trace[1250878695]: [627.771873ms] [627.771873ms] END E0628 20:12:16.532508 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 9.43µs, panicked: false, err: context deadline exceeded, panic-reason: E0628 20:12:16.532642 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:16.533708 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:16.533777 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:12:16.534972 1 trace.go:236] Trace[1359461509]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:74189e3d-75e0-4893-8873-3ffd6d167648,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (28-Jun-2026 20:12:11.532) (total time: 5001ms): Trace[1359461509]: ["GuaranteedUpdate etcd3" audit-id:74189e3d-75e0-4893-8873-3ffd6d167648,key:/leases/kube-system/kube-scheduler,type:*coordination.Lease,resource:leases.coordination.k8s.io 5001ms (20:12:11.533) Trace[1359461509]: ---"Txn call failed" err:context deadline exceeded 4998ms (20:12:16.532)] Trace[1359461509]: [5.001987334s] [5.001987334s] END E0628 20:12:16.535248 1 timeout.go:142] post-timeout activity - time-elapsed: 2.853825ms, PUT "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: E0628 20:12:17.365325 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0628 20:12:17.365417 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:17.366765 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:17.366844 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:12:17.367969 1 trace.go:236] Trace[256642352]: "Get" accept:application/json, */*,audit-id:e8cc274c-df26-4b73-9e96-726917013be3,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:12:12.365) (total time: 5002ms): Trace[256642352]: [5.002124027s] [5.002124027s] END E0628 20:12:17.368214 1 timeout.go:142] post-timeout activity - time-elapsed: 2.866264ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: E0628 20:12:18.618193 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:18.618389 1 trace.go:236] Trace[1501869703]: "Get" accept:application/json,audit-id:4189941b-10aa-4af9-be2f-e9a8df186c57,client:199.204.45.169,protocol:HTTP/2.0,resource:clusterroles,scope:resource,url:/apis/rbac.authorization.k8s.io/v1/clusterroles/local-path-provisioner,user-agent:kubectl/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:12:11.614) (total time: 7003ms): Trace[1501869703]: [7.003819188s] [7.003819188s] END E0628 20:12:19.146698 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0628 20:12:19.146851 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:19.148303 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:19.148357 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:12:19.149626 1 trace.go:236] Trace[435016189]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:176c03ed-2e67-4592-8248-e1bf9f038755,client:10.0.0.187,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:12:14.146) (total time: 5003ms): Trace[435016189]: [5.003039902s] [5.003039902s] END E0628 20:12:19.149891 1 timeout.go:142] post-timeout activity - time-elapsed: 3.096859ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com" result: E0628 20:12:20.390132 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0628 20:12:20.390302 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:20.391412 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:20.391538 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:12:20.392570 1 trace.go:236] Trace[603218584]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:d0475a7d-c8f9-4078-a45e-409d0b0196b2,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:12:15.390) (total time: 5001ms): Trace[603218584]: [5.001914081s] [5.001914081s] END E0628 20:12:20.392752 1 timeout.go:142] post-timeout activity - time-elapsed: 2.648831ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: E0628 20:12:21.215968 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0628 20:12:21.216090 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:21.217619 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:21.217732 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:12:21.218888 1 trace.go:236] Trace[963475456]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:59687c13-db27-4300-b549-f2798a0434d7,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:12:18.534) (total time: 2683ms): Trace[963475456]: [2.683871214s] [2.683871214s] END E0628 20:12:21.219147 1 timeout.go:142] post-timeout activity - time-elapsed: 3.12343ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: E0628 20:12:22.017827 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:22.018028 1 trace.go:236] Trace[603432357]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:e6bd23e3-5443-48f9-83a5-85e6264ff298,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:PUT (28-Jun-2026 20:12:15.014) (total time: 7003ms): Trace[603432357]: ["GuaranteedUpdate etcd3" audit-id:e6bd23e3-5443-48f9-83a5-85e6264ff298,key:/leases/ingress-nginx/ingress-nginx-leader,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (20:12:15.014) Trace[603432357]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (20:12:22.017)] Trace[603432357]: [7.003873279s] [7.003873279s] END E0628 20:12:22.266455 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0628 20:12:22.266604 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:22.267714 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:22.267797 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:12:22.269548 1 trace.go:236] Trace[417050966]: "Get" accept:application/json, */*,audit-id:d5e08dba-e3c3-435c-826d-20460eca8ef7,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:12:12.265) (total time: 10004ms): Trace[417050966]: [10.004161171s] [10.004161171s] END E0628 20:12:22.269731 1 timeout.go:142] post-timeout activity - time-elapsed: 3.379194ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock" result: E0628 20:12:22.270699 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0628 20:12:22.270923 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:22.271799 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:22.272043 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:12:22.273030 1 trace.go:236] Trace[1952337891]: "Get" accept:application/json, */*,audit-id:cd6e380b-11d0-442f-8478-d4cf468a99b6,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:12:12.270) (total time: 10002ms): Trace[1952337891]: [10.002483019s] [10.002483019s] END E0628 20:12:22.273268 1 timeout.go:142] post-timeout activity - time-elapsed: 2.485596ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock" result: E0628 20:12:22.365221 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0628 20:12:22.365348 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:22.366643 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:22.366861 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:12:22.368252 1 trace.go:236] Trace[1146569435]: "Get" accept:application/json, */*,audit-id:45c8d7f5-337d-4524-aefa-6d5d3bf35b5b,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:12:19.367) (total time: 3000ms): Trace[1146569435]: [3.000729369s] [3.000729369s] END E0628 20:12:22.368515 1 timeout.go:142] post-timeout activity - time-elapsed: 3.330253ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: E0628 20:12:24.615817 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:24.616100 1 trace.go:236] Trace[529280449]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:f9ab8238-1bb7-4a2b-91ed-382459f4152d,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Jun-2026 20:12:17.612) (total time: 7003ms): Trace[529280449]: ["GuaranteedUpdate etcd3" audit-id:f9ab8238-1bb7-4a2b-91ed-382459f4152d,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (20:12:17.612) Trace[529280449]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (20:12:24.615)] Trace[529280449]: [7.003876692s] [7.003876692s] END E0628 20:12:24.617229 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:24.617442 1 trace.go:236] Trace[454225817]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:90c982c0-dabf-4938-9eff-a07a624f6f73,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Jun-2026 20:12:17.613) (total time: 7003ms): Trace[454225817]: ["GuaranteedUpdate etcd3" audit-id:90c982c0-dabf-4938-9eff-a07a624f6f73,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (20:12:17.613) Trace[454225817]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (20:12:24.617)] Trace[454225817]: [7.003900712s] [7.003900712s] END E0628 20:12:24.618213 1 controller.go:193] "Failed to update lease" err="etcdserver: request timed out" I0628 20:12:25.619701 1 trace.go:236] Trace[1737429537]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.169,type:*v1.Endpoints,resource:apiServerIPInfo (28-Jun-2026 20:12:16.879) (total time: 8740ms): Trace[1737429537]: [8.740253938s] [8.740253938s] END E0628 20:12:25.619681 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:12:25.619758 1 controller.go:159] unable to sync kubernetes service: etcdserver: request timed out E0628 20:12:25.619729 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:12:25.619854 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:25.619946 1 trace.go:236] Trace[1868074706]: "Get" accept:application/json, */*,audit-id:fb4b594d-2f70-4206-a7d1-336a679dbba2,client:10.0.0.117,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-cainjector-leader-election,user-agent:cainjector/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:12:14.882) (total time: 10737ms): Trace[1868074706]: [10.737688989s] [10.737688989s] END E0628 20:12:25.619994 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:12:25.620006 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:12:25.620126 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:12:25.620201 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:12:25.620201 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:12:25.620844 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:25.620941 1 trace.go:236] Trace[1430573948]: "Get" accept:application/json, */*,audit-id:c1454fef-c660-47f1-b51f-6096ae3a78ac,client:10.0.0.139,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-controller,user-agent:cert-manager-leader-election/v1.12.17 (linux/amd64) cert-manager/37b853ff34a6c093e946c657c189f40413c0f628,verb:GET (28-Jun-2026 20:12:14.882) (total time: 10738ms): Trace[1430573948]: [10.738477783s] [10.738477783s] END I0628 20:12:25.620971 1 trace.go:236] Trace[1178754418]: "Get" accept:application/json, */*,audit-id:45699f79-c0c6-455e-899e-9799630e3bb0,client:10.0.0.225,protocol:HTTP/2.0,resource:nodefeatures,scope:resource,url:/apis/nfd.k8s-sigs.io/v1alpha1/namespaces/monitoring/nodefeatures/instance,user-agent:nfd-worker/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:12:15.444) (total time: 10176ms): Trace[1178754418]: [10.176887044s] [10.176887044s] END I0628 20:12:25.621261 1 trace.go:236] Trace[269853402]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:3b1f52df-d1ab-4ab5-bb4e-01551b9a889a,client:199.204.45.169,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/kube-system/events/kube-apiserver-instance.18bd572870f099b7,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (28-Jun-2026 20:12:13.666) (total time: 11955ms): Trace[269853402]: ["GuaranteedUpdate etcd3" audit-id:3b1f52df-d1ab-4ab5-bb4e-01551b9a889a,key:/events/kube-system/kube-apiserver-instance.18bd572870f099b7,type:*core.Event,resource:events 11954ms (20:12:13.666)] Trace[269853402]: [11.955000606s] [11.955000606s] END I0628 20:12:25.621367 1 trace.go:236] Trace[1500405822]: "Get" accept:application/json, */*,audit-id:5671ed14-6c98-44de-81c5-595814c84d6e,client:10.0.0.173,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:12:11.685) (total time: 13935ms): Trace[1500405822]: [13.935798732s] [13.935798732s] END I0628 20:12:25.621440 1 trace.go:236] Trace[1786819647]: "Get" accept:application/json,audit-id:073170f1-8e86-41e0-8cf5-7b77c2ec8af5,client:199.204.45.169,protocol:HTTP/2.0,resource:clusterroles,scope:resource,url:/apis/rbac.authorization.k8s.io/v1/clusterroles/kubeadm:get-nodes,user-agent:kubectl/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:12:11.616) (total time: 14004ms): Trace[1786819647]: [14.004463958s] [14.004463958s] END I0628 20:12:25.621460 1 trace.go:236] Trace[457024047]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:5b1dfd85-6e82-4fbe-bad8-07e3a26dd4cd,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/envoy-gateway-system/pods/envoy-gateway-78446f96c9-8mnfj,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:12:13.922) (total time: 11698ms): Trace[457024047]: [11.698823025s] [11.698823025s] END I0628 20:12:25.621467 1 trace.go:236] Trace[2531203]: "Get" accept:application/json, */*,audit-id:bf2d0ac1-1be3-4818-a795-4243d795f69e,client:10.0.0.25,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:12:12.607) (total time: 13013ms): Trace[2531203]: [13.013509015s] [13.013509015s] END I0628 20:12:25.621792 1 trace.go:236] Trace[1424147834]: "Get" accept:application/json, */*,audit-id:22e1df84-184c-4d4b-b8f7-52a554c86aeb,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/keepalived-qvhr9,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:12:12.907) (total time: 12713ms): Trace[1424147834]: [12.713778324s] [12.713778324s] END E0628 20:12:26.217482 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:26.217534 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:26.217627 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 25.421µs, panicked: false, err: context deadline exceeded, panic-reason: E0628 20:12:26.218822 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:12:26.218938 1 trace.go:236] Trace[1219428994]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:7c6b4c9c-c32e-4517-82fc-29622388b808,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (28-Jun-2026 20:12:21.217) (total time: 5001ms): Trace[1219428994]: ["GuaranteedUpdate etcd3" audit-id:7c6b4c9c-c32e-4517-82fc-29622388b808,key:/leases/kube-system/kube-scheduler,type:*coordination.Lease,resource:leases.coordination.k8s.io 5001ms (20:12:21.217) Trace[1219428994]: ---"Txn call failed" err:context deadline exceeded 4998ms (20:12:26.217)] Trace[1219428994]: [5.001371621s] [5.001371621s] END E0628 20:12:26.219219 1 timeout.go:142] post-timeout activity - time-elapsed: 1.863216ms, PUT "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: E0628 20:12:26.231174 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0628 20:12:26.231327 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:26.232381 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:26.232444 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:12:26.233545 1 trace.go:236] Trace[1268823756]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:e67eca19-b97a-4e13-853c-609dd3c57434,client:10.0.0.187,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:12:21.230) (total time: 5002ms): Trace[1268823756]: [5.0028869s] [5.0028869s] END E0628 20:12:26.233762 1 timeout.go:142] post-timeout activity - time-elapsed: 2.540569ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com" result: E0628 20:12:27.367566 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:27.367604 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:27.367633 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 12.5µs, panicked: false, err: context canceled, panic-reason: E0628 20:12:27.368786 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:12:27.368863 1 trace.go:236] Trace[1318397708]: "Update" accept:application/json, */*,audit-id:edd66256-a91c-48d4-b63a-55d06f1bf3da,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:12:22.369) (total time: 4999ms): Trace[1318397708]: ["GuaranteedUpdate etcd3" audit-id:edd66256-a91c-48d4-b63a-55d06f1bf3da,key:/leases/kube-system/cilium-operator-resource-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 4999ms (20:12:22.369) Trace[1318397708]: ---"Txn call failed" err:context canceled 4997ms (20:12:27.367)] Trace[1318397708]: [4.999793312s] [4.999793312s] END E0628 20:12:27.368964 1 timeout.go:142] post-timeout activity - time-elapsed: 1.392366ms, PUT "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: E0628 20:12:28.753099 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0628 20:12:28.753271 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:28.754384 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:28.754588 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:12:28.755544 1 trace.go:236] Trace[728883064]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:b7e2e0fa-75c0-41f2-83d8-2c3c0b3c303e,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:12:23.753) (total time: 5002ms): Trace[728883064]: [5.002196656s] [5.002196656s] END E0628 20:12:28.755817 1 timeout.go:142] post-timeout activity - time-elapsed: 2.603698ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: E0628 20:12:29.272749 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:29.273017 1 trace.go:236] Trace[475225041]: "Update" accept:application/json, */*,audit-id:03e0286d-525c-4519-9093-d6c2debd998d,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Jun-2026 20:12:22.268) (total time: 7004ms): Trace[475225041]: ["GuaranteedUpdate etcd3" audit-id:03e0286d-525c-4519-9093-d6c2debd998d,key:/leases/kube-system/plndr-cp-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 7004ms (20:12:22.268) Trace[475225041]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (20:12:29.272)] Trace[475225041]: [7.004604944s] [7.004604944s] END E0628 20:12:29.274754 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:29.275148 1 trace.go:236] Trace[622299858]: "Update" accept:application/json, */*,audit-id:09d31af0-65b5-4439-8aac-692063cf3b46,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Jun-2026 20:12:22.271) (total time: 7003ms): Trace[622299858]: ["GuaranteedUpdate etcd3" audit-id:09d31af0-65b5-4439-8aac-692063cf3b46,key:/leases/kube-system/plndr-svcs-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (20:12:22.272) Trace[622299858]: ---"Txn call failed" err:etcdserver: request timed out 7001ms (20:12:29.274)] Trace[622299858]: [7.00338058s] [7.00338058s] END E0628 20:12:30.013392 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0628 20:12:30.013520 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:30.015097 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:30.015196 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:12:30.016303 1 trace.go:236] Trace[809427923]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:6e272c48-3b23-4bd6-9d9e-fe44224a8124,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:GET (28-Jun-2026 20:12:22.020) (total time: 7995ms): Trace[809427923]: [7.995594298s] [7.995594298s] END E0628 20:12:30.016517 1 timeout.go:142] post-timeout activity - time-elapsed: 3.051318ms, GET "/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader" result: E0628 20:12:31.621664 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:31.621864 1 trace.go:236] Trace[2068583485]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:a6f5c05a-71b8-4529-a8ae-a5583061b8d2,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Jun-2026 20:12:24.618) (total time: 7002ms): Trace[2068583485]: ["GuaranteedUpdate etcd3" audit-id:a6f5c05a-71b8-4529-a8ae-a5583061b8d2,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 7002ms (20:12:24.619) Trace[2068583485]: ---"Txn call failed" err:etcdserver: request timed out 7001ms (20:12:31.621)] Trace[2068583485]: [7.002855299s] [7.002855299s] END E0628 20:12:31.622234 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:31.623916 1 trace.go:236] Trace[2056513782]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:448d161a-fb4d-4fef-800e-19c18291ce57,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Jun-2026 20:12:24.619) (total time: 7004ms): Trace[2056513782]: ["GuaranteedUpdate etcd3" audit-id:448d161a-fb4d-4fef-800e-19c18291ce57,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 7004ms (20:12:24.619) Trace[2056513782]: ---"Txn call failed" err:etcdserver: request timed out 7001ms (20:12:31.622)] Trace[2056513782]: [7.004492771s] [7.004492771s] END E0628 20:12:31.624579 1 controller.go:193] "Failed to update lease" err="etcdserver: request timed out" E0628 20:12:32.619437 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:32.619746 1 trace.go:236] Trace[668865954]: "Get" accept:application/json,audit-id:51f8df85-d703-40b3-8ee8-a74007f6d11b,client:199.204.45.169,protocol:HTTP/2.0,resource:clusterroles,scope:resource,url:/apis/rbac.authorization.k8s.io/v1/clusterroles/node-feature-discovery,user-agent:kubectl/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:12:18.701) (total time: 13917ms): Trace[668865954]: [13.917795049s] [13.917795049s] END E0628 20:12:34.503908 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:34.504153 1 trace.go:236] Trace[2081756499]: "Update" accept:application/json, */*,audit-id:b691f8c5-7854-4adf-b0b7-80a1548f03df,client:10.0.0.117,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-cainjector-leader-election,user-agent:cainjector/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Jun-2026 20:12:27.499) (total time: 7004ms): Trace[2081756499]: ["GuaranteedUpdate etcd3" audit-id:b691f8c5-7854-4adf-b0b7-80a1548f03df,key:/leases/cert-manager/cert-manager-cainjector-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (20:12:27.500) Trace[2081756499]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (20:12:34.503)] Trace[2081756499]: [7.004137134s] [7.004137134s] END E0628 20:12:34.516637 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:34.516677 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:34.516706 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 8.62µs, panicked: false, err: context canceled, panic-reason: E0628 20:12:34.517859 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout E0628 20:12:34.517997 1 timeout.go:142] post-timeout activity - time-elapsed: 1.356576ms, POST "/api/v1/namespaces/cert-manager/events" result: E0628 20:12:35.150761 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:35.151235 1 trace.go:236] Trace[1575215969]: "Update" accept:application/json, */*,audit-id:0199e66a-4dc2-4c45-9612-61c65d41f792,client:10.0.0.139,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-controller,user-agent:cert-manager-leader-election/v1.12.17 (linux/amd64) cert-manager/37b853ff34a6c093e946c657c189f40413c0f628,verb:PUT (28-Jun-2026 20:12:28.145) (total time: 7005ms): Trace[1575215969]: ["GuaranteedUpdate etcd3" audit-id:0199e66a-4dc2-4c45-9612-61c65d41f792,key:/leases/cert-manager/cert-manager-controller,type:*coordination.Lease,resource:leases.coordination.k8s.io 7004ms (20:12:28.147) Trace[1575215969]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (20:12:35.150)] Trace[1575215969]: [7.005175583s] [7.005175583s] END E0628 20:12:35.158449 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded I0628 20:12:35.158631 1 trace.go:236] Trace[588075315]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:b8762073-fa16-41f8-899d-28015a218e4b,client:10.0.0.187,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:12:30.158) (total time: 4999ms): Trace[588075315]: [4.999635078s] [4.999635078s] END E0628 20:12:35.158800 1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com?timeout=5s" audit-ID="b8762073-fa16-41f8-899d-28015a218e4b" E0628 20:12:35.158872 1 timeout.go:142] post-timeout activity - time-elapsed: 7.44µs, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com" result: E0628 20:12:35.164564 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:35.164608 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:35.164773 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 242.594µs, panicked: false, err: context canceled, panic-reason: E0628 20:12:35.166206 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout E0628 20:12:35.166308 1 timeout.go:142] post-timeout activity - time-elapsed: 1.761833ms, POST "/api/v1/namespaces/cert-manager/events" result: E0628 20:12:36.868721 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0628 20:12:36.869459 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:36.870090 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:36.871332 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:12:36.871422 1 trace.go:236] Trace[415973976]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:e696fde6-7c6c-4825-963b-ac09f76e1362,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:12:31.868) (total time: 5002ms): Trace[415973976]: [5.002654637s] [5.002654637s] END E0628 20:12:36.871760 1 timeout.go:142] post-timeout activity - time-elapsed: 2.336285ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: E0628 20:12:37.019665 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:37.019859 1 trace.go:236] Trace[1486376598]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:644980e3-13aa-4722-8be3-8acea1d11347,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:PUT (28-Jun-2026 20:12:30.014) (total time: 7004ms): Trace[1486376598]: ["GuaranteedUpdate etcd3" audit-id:644980e3-13aa-4722-8be3-8acea1d11347,key:/leases/ingress-nginx/ingress-nginx-leader,type:*coordination.Lease,resource:leases.coordination.k8s.io 7004ms (20:12:30.015) Trace[1486376598]: ---"Txn call failed" err:etcdserver: request timed out 7003ms (20:12:37.019)] Trace[1486376598]: [7.004957001s] [7.004957001s] END E0628 20:12:38.629347 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:12:38.629510 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:38.629564 1 trace.go:236] Trace[963137868]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:24395a94-73dc-43f4-b75e-daad9ac524b0,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Jun-2026 20:12:31.624) (total time: 7004ms): Trace[963137868]: ["GuaranteedUpdate etcd3" audit-id:24395a94-73dc-43f4-b75e-daad9ac524b0,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 7004ms (20:12:31.624) Trace[963137868]: ---"Txn call failed" err:etcdserver: request timed out 7003ms (20:12:38.629)] Trace[963137868]: [7.004751517s] [7.004751517s] END I0628 20:12:38.631197 1 trace.go:236] Trace[1031691858]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:633eb7f6-2a20-477d-8b66-578fe627aead,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Jun-2026 20:12:31.625) (total time: 7005ms): Trace[1031691858]: ["GuaranteedUpdate etcd3" audit-id:633eb7f6-2a20-477d-8b66-578fe627aead,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 7005ms (20:12:31.625) Trace[1031691858]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (20:12:38.629)] Trace[1031691858]: [7.005560853s] [7.005560853s] END E0628 20:12:38.631926 1 controller.go:193] "Failed to update lease" err="etcdserver: request timed out" I0628 20:12:39.620447 1 trace.go:236] Trace[745164456]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.169,type:*v1.Endpoints,resource:apiServerIPInfo (28-Jun-2026 20:12:26.880) (total time: 12740ms): Trace[745164456]: [12.740022863s] [12.740022863s] END E0628 20:12:39.620497 1 controller.go:159] unable to sync kubernetes service: etcdserver: request timed out E0628 20:12:39.620662 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:12:39.620663 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:12:39.620690 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:12:39.620700 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:12:39.620712 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:12:39.620749 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:39.622512 1 trace.go:236] Trace[1299961103]: "Get" accept:application/json, */*,audit-id:c9ccef7f-093d-4be6-90ff-8d1aa66ac7fb,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/keepalived-qvhr9,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:12:27.627) (total time: 11994ms): Trace[1299961103]: [11.994687351s] [11.994687351s] END I0628 20:12:39.622672 1 trace.go:236] Trace[30923929]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:164c921d-a477-4fcb-8e4f-d01317cfe8be,client:199.204.45.169,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/envoy-gateway-system/events/envoy-gateway-78446f96c9-8mnfj.18bd57c0fe4ce97f,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (28-Jun-2026 20:12:25.624) (total time: 13997ms): Trace[30923929]: ["GuaranteedUpdate etcd3" audit-id:164c921d-a477-4fcb-8e4f-d01317cfe8be,key:/events/envoy-gateway-system/envoy-gateway-78446f96c9-8mnfj.18bd57c0fe4ce97f,type:*core.Event,resource:events 13997ms (20:12:25.625)] Trace[30923929]: [13.997663408s] [13.997663408s] END I0628 20:12:39.622712 1 trace.go:236] Trace[37939568]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:73451b71-b236-48d5-b2bd-0a5ab4d05ee1,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/envoy-gateway-system/pods/envoy-gateway-78446f96c9-8mnfj,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:12:25.625) (total time: 13996ms): Trace[37939568]: [13.996898743s] [13.996898743s] END I0628 20:12:39.622750 1 trace.go:236] Trace[2083425290]: "Get" accept:application/json,audit-id:49c80bf4-54e8-49c5-b21b-35f616c3eab3,client:199.204.45.169,protocol:HTTP/2.0,resource:clusterroles,scope:resource,url:/apis/rbac.authorization.k8s.io/v1/clusterroles/node-feature-discovery-gc,user-agent:kubectl/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:12:25.704) (total time: 13918ms): Trace[2083425290]: [13.918289808s] [13.918289808s] END I0628 20:12:39.622777 1 trace.go:236] Trace[1829044437]: "Get" accept:application/json, */*,audit-id:afe0b316-d56b-4d02-8caa-240609f3fa87,client:10.0.0.25,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:12:28.552) (total time: 11070ms): Trace[1829044437]: [11.070136992s] [11.070136992s] END I0628 20:12:39.622726 1 trace.go:236] Trace[1675382282]: "Get" accept:application/json, */*,audit-id:6b1de9dc-548b-4faa-9bbe-841ac0263087,client:10.0.0.173,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:12:28.414) (total time: 11208ms): Trace[1675382282]: [11.208192476s] [11.208192476s] END I0628 20:12:40.785211 1 metrics.go:285] "Failed to get storage metrics" storage_cluster_id="etcd-0" err="context deadline exceeded" E0628 20:12:43.108072 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0628 20:12:43.108234 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:43.109474 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:43.109598 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:12:43.110859 1 trace.go:236] Trace[301909829]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:6726e7f5-75a6-4fa0-8ce5-3e5b55318498,client:10.0.0.187,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:12:38.108) (total time: 5002ms): Trace[301909829]: [5.00230972s] [5.00230972s] END E0628 20:12:43.111130 1 timeout.go:142] post-timeout activity - time-elapsed: 2.804533ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com" result: E0628 20:12:44.547603 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0628 20:12:44.547717 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:44.548872 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:44.548921 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:12:44.550141 1 trace.go:236] Trace[1231584492]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:3b37c18e-558b-4f7d-8f38-77c21a98d988,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:12:39.547) (total time: 5002ms): Trace[1231584492]: [5.002215159s] [5.002215159s] END E0628 20:12:44.550420 1 timeout.go:142] post-timeout activity - time-elapsed: 2.803843ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: E0628 20:12:45.635682 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:45.635892 1 trace.go:236] Trace[469173381]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:edb90f33-0cfc-49b0-81a7-eec02bc2a861,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Jun-2026 20:12:38.631) (total time: 7004ms): Trace[469173381]: ["GuaranteedUpdate etcd3" audit-id:edb90f33-0cfc-49b0-81a7-eec02bc2a861,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 7004ms (20:12:38.631) Trace[469173381]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (20:12:45.635)] Trace[469173381]: [7.004257308s] [7.004257308s] END E0628 20:12:45.635902 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:45.637127 1 trace.go:236] Trace[1303361115]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:e680e2ef-87e2-4025-8aed-6cb9a76bfc18,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Jun-2026 20:12:38.632) (total time: 7004ms): Trace[1303361115]: ["GuaranteedUpdate etcd3" audit-id:e680e2ef-87e2-4025-8aed-6cb9a76bfc18,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 7004ms (20:12:38.632) Trace[1303361115]: ---"Txn call failed" err:etcdserver: request timed out 7001ms (20:12:45.635)] Trace[1303361115]: [7.004229587s] [7.004229587s] END E0628 20:12:45.637921 1 controller.go:193] "Failed to update lease" err="etcdserver: request timed out" E0628 20:12:46.620506 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:46.620797 1 trace.go:236] Trace[774172131]: "Get" accept:application/json,audit-id:398c91f7-afb6-4370-9615-cc2fb92c9b5d,client:199.204.45.169,protocol:HTTP/2.0,resource:clusterroles,scope:resource,url:/apis/rbac.authorization.k8s.io/v1/clusterroles/node-feature-discovery,user-agent:kubectl/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:12:32.709) (total time: 13911ms): Trace[774172131]: [13.911018732s] [13.911018732s] END E0628 20:12:46.620941 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:46.622662 1 trace.go:236] Trace[2045129334]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:a30232d8-d37a-4cf1-8c31-b5327a98c981,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:GET (28-Jun-2026 20:12:37.022) (total time: 9600ms): Trace[2045129334]: [9.600403095s] [9.600403095s] END E0628 20:12:51.683259 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0628 20:12:51.683529 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:51.684878 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:51.684988 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:12:51.686109 1 trace.go:236] Trace[1459902760]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:4931b1b4-f2c4-47da-854f-edf1fc4327c9,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:12:46.683) (total time: 5002ms): Trace[1459902760]: [5.002728378s] [5.002728378s] END E0628 20:12:51.686369 1 timeout.go:142] post-timeout activity - time-elapsed: 2.824343ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: E0628 20:12:52.015453 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0628 20:12:52.015584 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:52.016931 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:52.016978 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:12:52.018208 1 trace.go:236] Trace[2135034176]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:58c25d4a-b2b3-46a6-82f6-c31f3a485329,client:10.0.0.187,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:12:47.015) (total time: 5002ms): Trace[2135034176]: [5.002144575s] [5.002144575s] END E0628 20:12:52.018533 1 timeout.go:142] post-timeout activity - time-elapsed: 2.779572ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com" result: E0628 20:12:52.642434 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:12:52.642493 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:52.642582 1 trace.go:236] Trace[1281792693]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:c2572e21-ea06-45d7-83df-67db19366d51,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Jun-2026 20:12:45.638) (total time: 7003ms): Trace[1281792693]: ["GuaranteedUpdate etcd3" audit-id:c2572e21-ea06-45d7-83df-67db19366d51,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (20:12:45.639) Trace[1281792693]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (20:12:52.642)] Trace[1281792693]: [7.003722572s] [7.003722572s] END I0628 20:12:52.644298 1 trace.go:236] Trace[1876404285]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:eccd55a1-38c4-4c4d-b8e0-c1a3416a59b0,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Jun-2026 20:12:45.638) (total time: 7005ms): Trace[1876404285]: ["GuaranteedUpdate etcd3" audit-id:eccd55a1-38c4-4c4d-b8e0-c1a3416a59b0,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 7005ms (20:12:45.639) Trace[1876404285]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (20:12:52.642)] Trace[1876404285]: [7.005300951s] [7.005300951s] END E0628 20:12:52.645058 1 controller.go:193] "Failed to update lease" err="etcdserver: request timed out" I0628 20:12:52.645100 1 controller.go:116] "failed to update lease using latest lease, fallback to ensure lease" err="failed 5 attempts to update lease" E0628 20:12:53.622070 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:53.622492 1 trace.go:236] Trace[1715042484]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.169,type:*v1.Endpoints,resource:apiServerIPInfo (28-Jun-2026 20:12:39.620) (total time: 14001ms): Trace[1715042484]: [14.00189297s] [14.00189297s] END E0628 20:12:53.622503 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:12:53.622527 1 controller.go:159] unable to sync kubernetes service: etcdserver: request timed out E0628 20:12:53.622528 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:12:53.622554 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:12:53.622574 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:12:53.622634 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:12:53.622647 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:12:53.622667 1 trace.go:236] Trace[235987078]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:17d60da0-5b72-4ee0-a3da-d36d95a454ff,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/kube-system/pods/kube-apiserver-instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:12:39.625) (total time: 13997ms): Trace[235987078]: [13.997062898s] [13.997062898s] END I0628 20:12:53.623357 1 trace.go:236] Trace[1933769015]: "Get" accept:application/json,audit-id:e024b3ce-f661-45be-8d52-efa151b3b69f,client:199.204.45.169,protocol:HTTP/2.0,resource:clusterroles,scope:resource,url:/apis/rbac.authorization.k8s.io/v1/clusterroles/node-feature-discovery-gc,user-agent:kubectl/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:12:39.702) (total time: 13920ms): Trace[1933769015]: [13.920944311s] [13.920944311s] END I0628 20:12:53.623839 1 trace.go:236] Trace[1065601283]: "Get" accept:application/json, */*,audit-id:b6e84656-5708-4ec0-aff5-6b5b0ec0db16,client:10.0.0.25,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:12:42.672) (total time: 10951ms): Trace[1065601283]: [10.951781732s] [10.951781732s] END I0628 20:12:53.623873 1 trace.go:236] Trace[420853834]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:78d31e16-3c18-4402-8f4a-c36482435b5a,client:199.204.45.169,protocol:HTTP/2.0,resource:persistentvolumeclaims,scope:resource,url:/api/v1/namespaces/openstack/persistentvolumeclaims/valkey-data-valkey-node-0,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:12:40.256) (total time: 13367ms): Trace[420853834]: [13.367387055s] [13.367387055s] END I0628 20:12:53.624006 1 trace.go:236] Trace[1618557449]: "Get" accept:application/json, */*,audit-id:2d144536-5e02-4439-9de5-2b1780c2196c,client:10.0.0.173,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:12:43.397) (total time: 10226ms): Trace[1618557449]: [10.226588302s] [10.226588302s] END I0628 20:12:53.624198 1 trace.go:236] Trace[2077480277]: "Get" accept:application/json, */*,audit-id:edaab856-a655-4ee4-98ae-dfd978c9dbf5,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/keepalived-qvhr9,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:12:41.626) (total time: 11997ms): Trace[2077480277]: [11.997707639s] [11.997707639s] END I0628 20:12:53.624227 1 trace.go:236] Trace[1603741694]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:6eded6d4-f768-41ec-ac7b-0f22a37c07df,client:199.204.45.169,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/envoy-gateway-system/events/envoy-gateway-78446f96c9-8mnfj.18bd57c0fe4ce97f,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (28-Jun-2026 20:12:39.625) (total time: 13998ms): Trace[1603741694]: ["GuaranteedUpdate etcd3" audit-id:6eded6d4-f768-41ec-ac7b-0f22a37c07df,key:/events/envoy-gateway-system/envoy-gateway-78446f96c9-8mnfj.18bd57c0fe4ce97f,type:*core.Event,resource:events 13998ms (20:12:39.626)] Trace[1603741694]: [13.998221681s] [13.998221681s] END E0628 20:12:59.052227 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0628 20:12:59.052395 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:59.054208 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:59.054258 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:12:59.055446 1 trace.go:236] Trace[473196733]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:4ba51c42-fcd2-4a63-9e5b-167e68e5768f,client:10.0.0.187,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:12:54.053) (total time: 5002ms): Trace[473196733]: [5.002340146s] [5.002340146s] END E0628 20:12:59.055667 1 timeout.go:142] post-timeout activity - time-elapsed: 3.18472ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com" result: E0628 20:12:59.417974 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0628 20:12:59.418158 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:12:59.419146 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:12:59.419329 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:12:59.420538 1 trace.go:236] Trace[1348554653]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:e6222758-c6e9-48fa-8391-7d809761c3e0,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:12:54.417) (total time: 5002ms): Trace[1348554653]: [5.002781105s] [5.002781105s] END E0628 20:12:59.420714 1 timeout.go:142] post-timeout activity - time-elapsed: 2.708961ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: E0628 20:13:00.621564 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0628 20:13:00.621773 1 trace.go:236] Trace[646508306]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:462f8abb-723f-40b6-a832-205cae7f7747,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:12:52.645) (total time: 7975ms): Trace[646508306]: [7.975762513s] [7.975762513s] END E0628 20:13:00.622124 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:13:00.622222 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0628 20:13:00.622500 1 controller.go:146] "Failed to ensure lease exists, will retry" err="etcdserver: request timed out" interval="200ms" I0628 20:13:00.622834 1 trace.go:236] Trace[1483583892]: "Get" accept:application/json,audit-id:0a976a45-3e6d-42b9-9e2f-0037df446d72,client:199.204.45.169,protocol:HTTP/2.0,resource:clusterroles,scope:resource,url:/apis/rbac.authorization.k8s.io/v1/clusterroles/rabbitmq-cluster-operator-openstack,user-agent:kubectl/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:12:46.725) (total time: 13897ms): Trace[1483583892]: [13.897383347s] [13.897383347s] END I0628 20:13:00.623378 1 trace.go:236] Trace[1333684348]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:152f6ac9-3fbe-414d-824a-b44c167e2018,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:12:52.644) (total time: 7978ms): Trace[1333684348]: [7.978678668s] [7.978678668s] END E0628 20:13:07.153960 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0628 20:13:07.154184 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0628 20:13:07.155921 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0628 20:13:07.156018 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0628 20:13:07.158039 1 trace.go:236] Trace[1726723641]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:ec28d999-89b2-481c-a723-e6f72ad944d0,client:10.0.0.187,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:13:02.153) (total time: 5004ms): Trace[1726723641]: [5.004401364s] [5.004401364s] END E0628 20:13:07.158206 1 timeout.go:142] post-timeout activity - time-elapsed: 4.21446ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com" result: I0628 20:13:07.354881 1 trace.go:236] Trace[402840815]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:32e8c736-ffae-4d1a-9dc2-dc217aa0d474,client:199.204.45.169,protocol:HTTP/2.0,resource:persistentvolumeclaims,scope:resource,url:/api/v1/namespaces/openstack/persistentvolumeclaims/datadir-percona-xtradb-pxc-0,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:12:53.626) (total time: 13728ms): Trace[402840815]: ---"About to write a response" 13727ms (20:13:07.354) Trace[402840815]: [13.72804303s] [13.72804303s] END I0628 20:13:07.355007 1 trace.go:236] Trace[2061381643]: "Get" accept:application/json,audit-id:61d7e553-1983-4ed5-a0e1-1eb44ccf5f2c,client:199.204.45.169,protocol:HTTP/2.0,resource:clusterroles,scope:resource,url:/apis/rbac.authorization.k8s.io/v1/clusterroles/rabbitmq-cluster-operator-openstack-admin,user-agent:kubectl/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:12:53.708) (total time: 13646ms): Trace[2061381643]: ---"About to write a response" 13645ms (20:13:07.354) Trace[2061381643]: [13.646168894s] [13.646168894s] END I0628 20:13:07.355571 1 trace.go:236] Trace[110898019]: "Get" accept:application/json, */*,audit-id:7e980284-93b7-4a87-8324-9e39aa088c1a,client:10.0.0.173,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:12:55.822) (total time: 11533ms): Trace[110898019]: ---"About to write a response" 11532ms (20:13:07.355) Trace[110898019]: [11.533166905s] [11.533166905s] END I0628 20:13:07.356368 1 trace.go:236] Trace[2044447344]: "Get" accept:application/json,audit-id:dc6e409a-d325-46c2-b2d9-37ac70c0b9cb,client:199.204.45.169,protocol:HTTP/2.0,resource:clusterroles,scope:resource,url:/apis/rbac.authorization.k8s.io/v1/clusterroles/rabbitmq-cluster-operator-openstack,user-agent:kubectl/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:13:00.704) (total time: 6651ms): Trace[2044447344]: ---"About to write a response" 6651ms (20:13:07.356) Trace[2044447344]: [6.65184868s] [6.65184868s] END I0628 20:13:07.356478 1 trace.go:236] Trace[195435809]: "Get" accept:application/json, */*,audit-id:e6093bfd-8d9c-4cc6-a5cd-dfcb9daef776,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/keepalived-qvhr9,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Jun-2026 20:12:55.628) (total time: 11728ms): Trace[195435809]: ---"About to write a response" 11727ms (20:13:07.355) Trace[195435809]: [11.728391137s] [11.728391137s] END I0628 20:13:07.356611 1 trace.go:236] Trace[1379923153]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:fb47df72-987b-416a-a4d9-4413c275131e,client:199.204.45.169,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/kube-system/pods/kube-controller-manager-instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:12:53.624) (total time: 13731ms): Trace[1379923153]: ---"About to write a response" 13731ms (20:13:07.356) Trace[1379923153]: [13.731725111s] [13.731725111s] END I0628 20:13:07.357464 1 trace.go:236] Trace[1846723841]: "Get" accept:application/json, */*,audit-id:a79ba028-1397-4496-95be-9d3e6bd190ab,client:10.0.0.25,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Jun-2026 20:12:56.370) (total time: 10986ms): Trace[1846723841]: ---"About to write a response" 10986ms (20:13:07.357) Trace[1846723841]: [10.986640222s] [10.986640222s] END I0628 20:13:07.357510 1 trace.go:236] Trace[1440246032]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:373597c9-8787-4c86-8193-0db82c318c6c,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Jun-2026 20:13:03.553) (total time: 3804ms): Trace[1440246032]: ---"About to write a response" 3804ms (20:13:07.357) Trace[1440246032]: [3.804378255s] [3.804378255s] END I0628 20:13:07.357524 1 trace.go:236] Trace[176969857]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:77573809-adef-45db-a467-a8e63187aef3,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:GET (28-Jun-2026 20:12:59.909) (total time: 7448ms): Trace[176969857]: ---"About to write a response" 7447ms (20:13:07.357) Trace[176969857]: [7.448067348s] [7.448067348s] END I0628 20:13:07.357589 1 trace.go:236] Trace[1151278563]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:b536e3e5-1f3b-4b20-ae91-20791be79803,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:13:00.824) (total time: 6533ms): Trace[1151278563]: ---"About to write a response" 6532ms (20:13:07.357) Trace[1151278563]: [6.53303493s] [6.53303493s] END I0628 20:13:07.357614 1 trace.go:236] Trace[587356165]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:a3f0a915-8c21-493b-941c-28d95709fb9f,client:199.204.45.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (28-Jun-2026 20:13:00.825) (total time: 6531ms): Trace[587356165]: ---"About to write a response" 6531ms (20:13:07.357) Trace[587356165]: [6.531902109s] [6.531902109s] END I0628 20:13:07.359225 1 trace.go:236] Trace[1621409893]: "List" accept:application/json, */*,audit-id:03b19d24-e4ab-40ea-bfa9-fd50d70abef1,client:10.0.0.7,protocol:HTTP/2.0,resource:nodes,scope:cluster,url:/api/v1/nodes,user-agent:PrometheusOperator/0.74.0,verb:LIST (28-Jun-2026 20:13:05.420) (total time: 1938ms): Trace[1621409893]: ["List(recursive=true) etcd3" audit-id:03b19d24-e4ab-40ea-bfa9-fd50d70abef1,key:/minions,resourceVersion:,resourceVersionMatch:,limit:0,continue: 1938ms (20:13:05.420)] Trace[1621409893]: [1.938472786s] [1.938472786s] END I0628 20:13:07.368774 1 trace.go:236] Trace[228092806]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.169,type:*v1.Endpoints,resource:apiServerIPInfo (28-Jun-2026 20:12:53.623) (total time: 13745ms): Trace[228092806]: ---"initial value restored" 13731ms (20:13:07.355) Trace[228092806]: [13.745065574s] [13.745065574s] END I0628 20:13:07.371696 1 trace.go:236] Trace[791025776]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:83681316-bb7b-4def-8542-bf26c74dc312,client:199.204.45.169,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/openstack/events/rabbitmq-barbican-server-0.18bd57c19c8ebb04,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (28-Jun-2026 20:12:53.626) (total time: 13745ms): Trace[791025776]: ["GuaranteedUpdate etcd3" audit-id:83681316-bb7b-4def-8542-bf26c74dc312,key:/events/openstack/rabbitmq-barbican-server-0.18bd57c19c8ebb04,type:*core.Event,resource:events 13745ms (20:12:53.626) Trace[791025776]: ---"initial value restored" 13729ms (20:13:07.355)] Trace[791025776]: ---"Object stored in database" 14ms (20:13:07.371) Trace[791025776]: [13.745525454s] [13.745525454s] END