I0702 00:23:50.066978 1 options.go:220] external host was not specified, using 199.204.45.248 I0702 00:23:50.067663 1 server.go:148] Version: v1.28.13 I0702 00:23:50.067714 1 server.go:150] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" I0702 00:23:50.369552 1 shared_informer.go:311] Waiting for caches to sync for node_authorizer I0702 00:23:50.377260 1 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook. I0702 00:23:50.377292 1 plugins.go:161] Loaded 13 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,ClusterTrustBundleAttest,CertificateSubjectRestriction,ValidatingAdmissionPolicy,ValidatingAdmissionWebhook,ResourceQuota. I0702 00:23:50.377553 1 instance.go:298] Using reconciler: lease I0702 00:23:50.537428 1 handler.go:275] Adding GroupVersion apiextensions.k8s.io v1 to ResourceManager W0702 00:23:50.537451 1 genericapiserver.go:744] Skipping API apiextensions.k8s.io/v1beta1 because it has no resources. I0702 00:23:50.794276 1 handler.go:275] Adding GroupVersion v1 to ResourceManager I0702 00:23:50.794582 1 instance.go:709] API group "internal.apiserver.k8s.io" is not enabled, skipping. I0702 00:23:51.217643 1 instance.go:709] API group "resource.k8s.io" is not enabled, skipping. I0702 00:23:51.227776 1 handler.go:275] Adding GroupVersion authentication.k8s.io v1 to ResourceManager W0702 00:23:51.227873 1 genericapiserver.go:744] Skipping API authentication.k8s.io/v1beta1 because it has no resources. W0702 00:23:51.227895 1 genericapiserver.go:744] Skipping API authentication.k8s.io/v1alpha1 because it has no resources. I0702 00:23:51.228435 1 handler.go:275] Adding GroupVersion authorization.k8s.io v1 to ResourceManager W0702 00:23:51.228500 1 genericapiserver.go:744] Skipping API authorization.k8s.io/v1beta1 because it has no resources. I0702 00:23:51.229355 1 handler.go:275] Adding GroupVersion autoscaling v2 to ResourceManager I0702 00:23:51.230204 1 handler.go:275] Adding GroupVersion autoscaling v1 to ResourceManager W0702 00:23:51.230248 1 genericapiserver.go:744] Skipping API autoscaling/v2beta1 because it has no resources. W0702 00:23:51.230265 1 genericapiserver.go:744] Skipping API autoscaling/v2beta2 because it has no resources. I0702 00:23:51.231737 1 handler.go:275] Adding GroupVersion batch v1 to ResourceManager W0702 00:23:51.231787 1 genericapiserver.go:744] Skipping API batch/v1beta1 because it has no resources. I0702 00:23:51.232627 1 handler.go:275] Adding GroupVersion certificates.k8s.io v1 to ResourceManager W0702 00:23:51.232671 1 genericapiserver.go:744] Skipping API certificates.k8s.io/v1beta1 because it has no resources. W0702 00:23:51.232687 1 genericapiserver.go:744] Skipping API certificates.k8s.io/v1alpha1 because it has no resources. I0702 00:23:51.233207 1 handler.go:275] Adding GroupVersion coordination.k8s.io v1 to ResourceManager W0702 00:23:51.233248 1 genericapiserver.go:744] Skipping API coordination.k8s.io/v1beta1 because it has no resources. W0702 00:23:51.233303 1 genericapiserver.go:744] Skipping API discovery.k8s.io/v1beta1 because it has no resources. I0702 00:23:51.233859 1 handler.go:275] Adding GroupVersion discovery.k8s.io v1 to ResourceManager I0702 00:23:51.235561 1 handler.go:275] Adding GroupVersion networking.k8s.io v1 to ResourceManager W0702 00:23:51.235606 1 genericapiserver.go:744] Skipping API networking.k8s.io/v1beta1 because it has no resources. W0702 00:23:51.235624 1 genericapiserver.go:744] Skipping API networking.k8s.io/v1alpha1 because it has no resources. I0702 00:23:51.236035 1 handler.go:275] Adding GroupVersion node.k8s.io v1 to ResourceManager W0702 00:23:51.236076 1 genericapiserver.go:744] Skipping API node.k8s.io/v1beta1 because it has no resources. W0702 00:23:51.236093 1 genericapiserver.go:744] Skipping API node.k8s.io/v1alpha1 because it has no resources. I0702 00:23:51.236828 1 handler.go:275] Adding GroupVersion policy v1 to ResourceManager W0702 00:23:51.236872 1 genericapiserver.go:744] Skipping API policy/v1beta1 because it has no resources. I0702 00:23:51.238569 1 handler.go:275] Adding GroupVersion rbac.authorization.k8s.io v1 to ResourceManager W0702 00:23:51.238610 1 genericapiserver.go:744] Skipping API rbac.authorization.k8s.io/v1beta1 because it has no resources. W0702 00:23:51.238630 1 genericapiserver.go:744] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources. I0702 00:23:51.239037 1 handler.go:275] Adding GroupVersion scheduling.k8s.io v1 to ResourceManager W0702 00:23:51.239078 1 genericapiserver.go:744] Skipping API scheduling.k8s.io/v1beta1 because it has no resources. W0702 00:23:51.239094 1 genericapiserver.go:744] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources. I0702 00:23:51.240991 1 handler.go:275] Adding GroupVersion storage.k8s.io v1 to ResourceManager W0702 00:23:51.241035 1 genericapiserver.go:744] Skipping API storage.k8s.io/v1beta1 because it has no resources. W0702 00:23:51.241052 1 genericapiserver.go:744] Skipping API storage.k8s.io/v1alpha1 because it has no resources. I0702 00:23:51.242283 1 handler.go:275] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta3 to ResourceManager I0702 00:23:51.243322 1 handler.go:275] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta2 to ResourceManager W0702 00:23:51.243368 1 genericapiserver.go:744] Skipping API flowcontrol.apiserver.k8s.io/v1beta1 because it has no resources. W0702 00:23:51.243386 1 genericapiserver.go:744] Skipping API flowcontrol.apiserver.k8s.io/v1alpha1 because it has no resources. I0702 00:23:51.246852 1 handler.go:275] Adding GroupVersion apps v1 to ResourceManager W0702 00:23:51.246902 1 genericapiserver.go:744] Skipping API apps/v1beta2 because it has no resources. W0702 00:23:51.246920 1 genericapiserver.go:744] Skipping API apps/v1beta1 because it has no resources. I0702 00:23:51.247720 1 handler.go:275] Adding GroupVersion admissionregistration.k8s.io v1 to ResourceManager W0702 00:23:51.247764 1 genericapiserver.go:744] Skipping API admissionregistration.k8s.io/v1beta1 because it has no resources. W0702 00:23:51.247781 1 genericapiserver.go:744] Skipping API admissionregistration.k8s.io/v1alpha1 because it has no resources. I0702 00:23:51.248292 1 handler.go:275] Adding GroupVersion events.k8s.io v1 to ResourceManager W0702 00:23:51.248335 1 genericapiserver.go:744] Skipping API events.k8s.io/v1beta1 because it has no resources. I0702 00:23:51.281843 1 handler.go:275] Adding GroupVersion apiregistration.k8s.io v1 to ResourceManager W0702 00:23:51.281978 1 genericapiserver.go:744] Skipping API apiregistration.k8s.io/v1beta1 because it has no resources. I0702 00:23:51.680810 1 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/etc/kubernetes/pki/front-proxy-ca.crt" I0702 00:23:51.680915 1 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt" I0702 00:23:51.681177 1 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/etc/kubernetes/pki/apiserver.crt::/etc/kubernetes/pki/apiserver.key" I0702 00:23:51.681557 1 secure_serving.go:213] Serving securely on [::]:6443 I0702 00:23:51.681617 1 tlsconfig.go:240] "Starting DynamicServingCertificateController" I0702 00:23:51.681865 1 customresource_discovery_controller.go:289] Starting DiscoveryController I0702 00:23:51.682201 1 controller.go:116] Starting legacy_token_tracking_controller I0702 00:23:51.682238 1 shared_informer.go:311] Waiting for caches to sync for configmaps I0702 00:23:51.682381 1 aggregator.go:164] waiting for initial CRD sync... I0702 00:23:51.682423 1 handler_discovery.go:412] Starting ResourceDiscoveryManager I0702 00:23:51.682589 1 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller I0702 00:23:51.682623 1 shared_informer.go:311] Waiting for caches to sync for cluster_authentication_trust_controller I0702 00:23:51.682685 1 system_namespaces_controller.go:67] Starting system namespaces controller I0702 00:23:51.682919 1 gc_controller.go:78] Starting apiserver lease garbage collector I0702 00:23:51.682970 1 apiservice_controller.go:97] Starting APIServiceRegistrationController I0702 00:23:51.683009 1 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller I0702 00:23:51.683040 1 controller.go:80] Starting OpenAPI V3 AggregationController I0702 00:23:51.683742 1 apf_controller.go:374] Starting API Priority and Fairness config controller I0702 00:23:51.683859 1 dynamic_serving_content.go:132] "Starting controller" name="aggregator-proxy-cert::/etc/kubernetes/pki/front-proxy-client.crt::/etc/kubernetes/pki/front-proxy-client.key" I0702 00:23:51.683984 1 available_controller.go:423] Starting AvailableConditionController I0702 00:23:51.684029 1 cache.go:32] Waiting for caches to sync for AvailableConditionController controller I0702 00:23:51.684079 1 controller.go:78] Starting OpenAPI AggregationController I0702 00:23:51.684272 1 gc_controller.go:78] Starting apiserver lease garbage collector I0702 00:23:51.686776 1 controller.go:85] Starting OpenAPI V3 controller I0702 00:23:51.686847 1 naming_controller.go:291] Starting NamingConditionController I0702 00:23:51.686875 1 establishing_controller.go:76] Starting EstablishingController I0702 00:23:51.686895 1 nonstructuralschema_controller.go:192] Starting NonStructuralSchemaConditionController I0702 00:23:51.686915 1 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController I0702 00:23:51.686937 1 crd_finalizer.go:266] Starting CRDFinalizer I0702 00:23:51.687362 1 crdregistration_controller.go:111] Starting crd-autoregister controller I0702 00:23:51.687373 1 shared_informer.go:311] Waiting for caches to sync for crd-autoregister I0702 00:23:51.687426 1 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt" I0702 00:23:51.687615 1 controller.go:134] Starting OpenAPI controller I0702 00:23:51.687696 1 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/etc/kubernetes/pki/front-proxy-ca.crt" I0702 00:23:51.770498 1 shared_informer.go:318] Caches are synced for node_authorizer E0702 00:23:51.779341 1 controller.go:146] "Failed to ensure lease exists, will retry" err="namespaces \"kube-system\" not found" interval="200ms" I0702 00:23:51.782969 1 shared_informer.go:318] Caches are synced for configmaps I0702 00:23:51.783095 1 cache.go:39] Caches are synced for APIServiceRegistrationController controller I0702 00:23:51.783181 1 shared_informer.go:318] Caches are synced for cluster_authentication_trust_controller I0702 00:23:51.784198 1 cache.go:39] Caches are synced for AvailableConditionController controller I0702 00:23:51.784317 1 apf_controller.go:379] Running API Priority and Fairness config worker I0702 00:23:51.784335 1 apf_controller.go:382] Running API Priority and Fairness periodic rebalancing process I0702 00:23:51.784512 1 controller.go:624] quota admission added evaluator for: namespaces I0702 00:23:51.788000 1 shared_informer.go:318] Caches are synced for crd-autoregister I0702 00:23:51.788131 1 aggregator.go:166] initial CRD sync complete... I0702 00:23:51.788154 1 autoregister_controller.go:141] Starting autoregister controller I0702 00:23:51.788163 1 cache.go:32] Waiting for caches to sync for autoregister controller I0702 00:23:51.788171 1 cache.go:39] Caches are synced for autoregister controller I0702 00:23:51.984088 1 controller.go:624] quota admission added evaluator for: leases.coordination.k8s.io I0702 00:23:52.802407 1 storage_scheduling.go:95] created PriorityClass system-node-critical with value 2000001000 I0702 00:23:52.851357 1 storage_scheduling.go:95] created PriorityClass system-cluster-critical with value 2000000000 I0702 00:23:52.851385 1 storage_scheduling.go:111] all system priority classes are created successfully or already exist. I0702 00:23:53.366654 1 controller.go:624] quota admission added evaluator for: roles.rbac.authorization.k8s.io I0702 00:23:53.410563 1 controller.go:624] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io I0702 00:23:53.496842 1 alloc.go:330] "allocated clusterIPs" service="default/kubernetes" clusterIPs={"IPv4":"10.96.0.1"} W0702 00:23:53.505109 1 lease.go:263] Resetting endpoints for master service "kubernetes" to [199.204.45.248] I0702 00:23:53.506554 1 controller.go:624] quota admission added evaluator for: endpoints I0702 00:23:53.514099 1 controller.go:624] quota admission added evaluator for: endpointslices.discovery.k8s.io E0702 00:23:55.333153 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0702 00:23:55.333172 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 14.37µs, panicked: false, err: context canceled, panic-reason: E0702 00:23:55.333206 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0702 00:23:55.334998 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout E0702 00:23:55.335156 1 timeout.go:142] post-timeout activity - time-elapsed: 2.203762ms, POST "/api/v1/namespaces/kube-system/pods" result: I0702 00:23:55.918052 1 controller.go:624] quota admission added evaluator for: serviceaccounts I0702 00:23:55.931570 1 controller.go:624] quota admission added evaluator for: deployments.apps I0702 00:23:55.949047 1 alloc.go:330] "allocated clusterIPs" service="kube-system/kube-dns" clusterIPs={"IPv4":"10.96.0.10"} I0702 00:23:55.962812 1 controller.go:624] quota admission added evaluator for: daemonsets.apps I0702 00:24:00.620887 1 controller.go:624] quota admission added evaluator for: replicasets.apps I0702 00:24:00.868936 1 controller.go:624] quota admission added evaluator for: controllerrevisions.apps I0702 00:24:25.654238 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0702 00:24:25.654361 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0702 00:24:25.733982 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0702 00:24:25.755800 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0702 00:24:25.768503 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0702 00:24:25.784201 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0702 00:24:25.803845 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0702 00:24:25.844787 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0702 00:24:25.845547 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha3 to ResourceManager I0702 00:24:25.869196 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0702 00:24:25.902719 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0702 00:24:25.902757 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0702 00:24:25.942037 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0702 00:24:26.105027 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0702 00:24:26.105119 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0702 00:24:26.113053 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0702 00:24:26.113094 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha3 to ResourceManager I0702 00:24:26.134978 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0702 00:24:26.173381 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0702 00:24:26.192832 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0702 00:24:26.218870 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0702 00:24:26.232019 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0702 00:24:26.787872 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0702 00:24:26.799570 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0702 00:24:26.869488 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0702 00:24:32.515560 1 controller.go:624] quota admission added evaluator for: jobs.batch I0702 00:24:33.385258 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0702 00:24:33.390399 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0702 00:24:33.399559 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0702 00:24:33.567831 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0702 00:24:33.811256 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0702 00:24:33.953794 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0702 00:24:34.349287 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0702 00:24:35.958560 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0702 00:24:36.575002 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0702 00:24:37.000551 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0702 00:24:37.345586 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager W0702 00:24:42.332652 1 dispatcher.go:210] Failed calling webhook, failing open topology.webhook.gateway.envoyproxy.io: failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": service "envoy-gateway" not found E0702 00:24:42.332687 1 dispatcher.go:214] failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": service "envoy-gateway" not found I0702 00:24:42.804625 1 controller.go:624] quota admission added evaluator for: ciliumendpoints.cilium.io I0702 00:24:50.769149 1 alloc.go:330] "allocated clusterIPs" service="envoy-gateway-system/envoy-gateway" clusterIPs={"IPv4":"10.104.137.224"} W0702 00:24:50.791967 1 dispatcher.go:210] Failed calling webhook, failing open topology.webhook.gateway.envoyproxy.io: failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": dial tcp 10.104.137.224:9443: connect: connection refused E0702 00:24:50.792010 1 dispatcher.go:214] failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": dial tcp 10.104.137.224:9443: connect: connection refused I0702 00:25:57.707697 1 alloc.go:330] "allocated clusterIPs" service="kube-system/ceph-csi-rbd-nodeplugin-http-metrics" clusterIPs={"IPv4":"10.101.190.37"} I0702 00:25:57.715944 1 alloc.go:330] "allocated clusterIPs" service="kube-system/ceph-csi-rbd-provisioner-http-metrics" clusterIPs={"IPv4":"10.111.71.163"}