2026-03-05 16:14:04.988489 | Job console starting
2026-03-05 16:14:05.020140 | Updating git repos
2026-03-05 16:14:05.054241 | Cloning repos into workspace
2026-03-05 16:14:05.178584 | Restoring repo states
2026-03-05 16:14:05.190945 | Merging changes
2026-03-05 16:14:05.798489 | Checking out repos
2026-03-05 16:14:06.043792 | Preparing playbooks
2026-03-05 16:14:07.369584 | Running Ansible setup
2026-03-05 16:14:16.954232 | PRE-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main]
2026-03-05 16:14:17.631749 | 
2026-03-05 16:14:17.632213 | PLAY [localhost]
2026-03-05 16:14:17.639948 | 
2026-03-05 16:14:17.640020 | TASK [Gathering Facts]
2026-03-05 16:14:18.537669 | localhost | ok
2026-03-05 16:14:18.544473 | 
2026-03-05 16:14:18.544551 | TASK [Setup log path fact]
2026-03-05 16:14:18.568622 | localhost | ok
2026-03-05 16:14:18.583387 | 
2026-03-05 16:14:18.583459 | TASK [set-zuul-log-path-fact : Set log path for a build]
2026-03-05 16:14:18.615549 | localhost | ok
2026-03-05 16:14:18.627356 | 
2026-03-05 16:14:18.627452 | TASK [emit-job-header : Print job information]
2026-03-05 16:14:18.669956 | # Job Information
2026-03-05 16:14:18.670165 | Ansible Version: 2.16.16
2026-03-05 16:14:18.670234 | Job: ansible-collection-kubernetes-molecule-cluster-upgrade
2026-03-05 16:14:18.670284 | Pipeline: check
2026-03-05 16:14:18.670330 | Executor: 0a8996d2b663
2026-03-05 16:14:18.670374 | Triggered by: https://github.com/vexxhost/ansible-collection-kubernetes/pull/234
2026-03-05 16:14:18.670423 | Event ID: f483c040-18ad-11f1-82de-983357ded420
2026-03-05 16:14:18.675624 | 
2026-03-05 16:14:18.675821 | LOOP [emit-job-header : Print node information]
2026-03-05 16:14:18.772218 | localhost | ok:
2026-03-05 16:14:18.772667 | localhost | # Node Information
2026-03-05 16:14:18.772740 | localhost | Inventory Hostname: controller-1
2026-03-05 16:14:18.772792 | localhost | Hostname: np0000157878
2026-03-05 16:14:18.772841 | localhost | Username: zuul
2026-03-05 16:14:18.772904 | localhost | Distro: Debian 13.3
2026-03-05 16:14:18.772982 | localhost | Provider: yul1
2026-03-05 16:14:18.773034 | localhost | Region: ca-ymq-1
2026-03-05 16:14:18.773080 | localhost | Label: debian-trixie
2026-03-05 16:14:18.773124 | localhost | Product Name: OpenStack Nova
2026-03-05 16:14:18.773169 | localhost | Interface IP: 162.253.55.206
2026-03-05 16:14:18.805010 | localhost | ok:
2026-03-05 16:14:18.805327 | localhost | # Node Information
2026-03-05 16:14:18.805402 | localhost | Inventory Hostname: controller-2
2026-03-05 16:14:18.805453 | localhost | Hostname: np0000157879
2026-03-05 16:14:18.805498 | localhost | Username: zuul
2026-03-05 16:14:18.805540 | localhost | Distro: Debian 13.3
2026-03-05 16:14:18.805582 | localhost | Provider: yul1
2026-03-05 16:14:18.805623 | localhost | Region: ca-ymq-1
2026-03-05 16:14:18.805663 | localhost | Label: debian-trixie
2026-03-05 16:14:18.805705 | localhost | Product Name: OpenStack Nova
2026-03-05 16:14:18.805926 | localhost | Interface IP: 199.204.45.156
2026-03-05 16:14:18.845059 | localhost | ok:
2026-03-05 16:14:18.845210 | localhost | # Node Information
2026-03-05 16:14:18.845239 | localhost | Inventory Hostname: controller-3
2026-03-05 16:14:18.845260 | localhost | Hostname: np0000157880
2026-03-05 16:14:18.845280 | localhost | Username: zuul
2026-03-05 16:14:18.845300 | localhost | Distro: Debian 13.3
2026-03-05 16:14:18.845319 | localhost | Provider: yul1
2026-03-05 16:14:18.845337 | localhost | Region: ca-ymq-1
2026-03-05 16:14:18.845355 | localhost | Label: debian-trixie
2026-03-05 16:14:18.845373 | localhost | Product Name: OpenStack Nova
2026-03-05 16:14:18.845391 | localhost | Interface IP: 199.204.45.244
2026-03-05 16:14:18.860166 | 
2026-03-05 16:14:18.860313 | TASK [log-inventory : Ensure Zuul Ansible directory exists]
2026-03-05 16:14:19.251506 | localhost -> localhost | changed
2026-03-05 16:14:19.260501 | 
2026-03-05 16:14:19.260606 | TASK [log-inventory : Copy ansible inventory to logs dir]
2026-03-05 16:14:20.168608 | localhost -> localhost | changed
2026-03-05 16:14:20.186363 | 
2026-03-05 16:14:20.186512 | PLAY [all]
2026-03-05 16:14:20.198533 | 
2026-03-05 16:14:20.198621 | TASK [add-build-sshkey : Check to see if ssh key was already created for this build]
2026-03-05 16:14:20.441726 | controller-1 -> localhost | ok
2026-03-05 16:14:20.451232 | 
2026-03-05 16:14:20.451357 | TASK [add-build-sshkey : Create a new key in workspace based on build UUID]
2026-03-05 16:14:20.489700 | controller-1 | ok
2026-03-05 16:14:20.493032 | controller-2 | ok
2026-03-05 16:14:20.503478 | controller-3 | ok
2026-03-05 16:14:20.522787 | controller-1 | included: /var/lib/zuul/builds/3f7a9e4c321043949fd99e3d8a4500aa/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml
2026-03-05 16:14:20.522836 | controller-2 | included: /var/lib/zuul/builds/3f7a9e4c321043949fd99e3d8a4500aa/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml
2026-03-05 16:14:20.522861 | controller-3 | included: /var/lib/zuul/builds/3f7a9e4c321043949fd99e3d8a4500aa/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml
2026-03-05 16:14:20.528859 | 
2026-03-05 16:14:20.528922 | TASK [add-build-sshkey : Create Temp SSH key]
2026-03-05 16:14:21.595161 | controller-1 -> localhost | Generating public/private rsa key pair.
2026-03-05 16:14:21.595395 | controller-1 -> localhost | Your identification has been saved in /var/lib/zuul/builds/3f7a9e4c321043949fd99e3d8a4500aa/work/3f7a9e4c321043949fd99e3d8a4500aa_id_rsa
2026-03-05 16:14:21.595429 | controller-1 -> localhost | Your public key has been saved in /var/lib/zuul/builds/3f7a9e4c321043949fd99e3d8a4500aa/work/3f7a9e4c321043949fd99e3d8a4500aa_id_rsa.pub
2026-03-05 16:14:21.595453 | controller-1 -> localhost | The key fingerprint is:
2026-03-05 16:14:21.595475 | controller-1 -> localhost | SHA256:3ZJ8cpAblm5/rt3lA8c2l0yND0c19kXQ3ckudaJYPFc zuul-build-sshkey
2026-03-05 16:14:21.595496 | controller-1 -> localhost | The key's randomart image is:
2026-03-05 16:14:21.595517 | controller-1 -> localhost | +---[RSA 3072]----+
2026-03-05 16:14:21.595537 | controller-1 -> localhost | |           .  oBE|
2026-03-05 16:14:21.595558 | controller-1 -> localhost | |           o+ ++X|
2026-03-05 16:14:21.595578 | controller-1 -> localhost | |          *o +o++|
2026-03-05 16:14:21.595598 | controller-1 -> localhost | |         =.*..ooo|
2026-03-05 16:14:21.595618 | controller-1 -> localhost | |        S X + =+.|
2026-03-05 16:14:21.595637 | controller-1 -> localhost | |         . * . Bo|
2026-03-05 16:14:21.595656 | controller-1 -> localhost | |            . = +|
2026-03-05 16:14:21.595676 | controller-1 -> localhost | |             + +.|
2026-03-05 16:14:21.595695 | controller-1 -> localhost | |            ..o +|
2026-03-05 16:14:21.595715 | controller-1 -> localhost | +----[SHA256]-----+
2026-03-05 16:14:21.595761 | controller-1 -> localhost | ok: Runtime: 0:00:00.632770
2026-03-05 16:14:21.600111 | 
2026-03-05 16:14:21.600171 | TASK [add-build-sshkey : Remote setup ssh keys (linux)]
2026-03-05 16:14:21.633472 | controller-1 | ok
2026-03-05 16:14:21.638473 | controller-2 | ok
2026-03-05 16:14:21.656284 | controller-3 | ok
2026-03-05 16:14:21.669635 | controller-1 | included: /var/lib/zuul/builds/3f7a9e4c321043949fd99e3d8a4500aa/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml
2026-03-05 16:14:21.669684 | controller-2 | included: /var/lib/zuul/builds/3f7a9e4c321043949fd99e3d8a4500aa/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml
2026-03-05 16:14:21.669708 | controller-3 | included: /var/lib/zuul/builds/3f7a9e4c321043949fd99e3d8a4500aa/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml
2026-03-05 16:14:21.681262 | 
2026-03-05 16:14:21.681330 | TASK [add-build-sshkey : Remove previously added zuul-build-sshkey]
2026-03-05 16:14:21.703972 | controller-1 | skipping: Conditional result was False
2026-03-05 16:14:21.705957 | controller-2 | skipping: Conditional result was False
2026-03-05 16:14:21.716376 | controller-3 | skipping: Conditional result was False
2026-03-05 16:14:21.724534 | 
2026-03-05 16:14:21.724623 | TASK [add-build-sshkey : Enable access via build key on all nodes]
2026-03-05 16:14:22.270348 | controller-1 | changed
2026-03-05 16:14:22.285601 | controller-2 | changed
2026-03-05 16:14:22.291342 | controller-3 | changed
2026-03-05 16:14:22.295756 | 
2026-03-05 16:14:22.295817 | TASK [add-build-sshkey : Make sure user has a .ssh]
2026-03-05 16:14:22.516120 | controller-1 | ok
2026-03-05 16:14:22.534879 | controller-2 | ok
2026-03-05 16:14:22.540794 | controller-3 | ok
2026-03-05 16:14:22.545165 | 
2026-03-05 16:14:22.545226 | TASK [add-build-sshkey : Install build private key as SSH key on all nodes]
2026-03-05 16:14:23.093745 | controller-3 | changed
2026-03-05 16:14:23.099358 | controller-2 | changed
2026-03-05 16:14:23.114391 | controller-1 | changed
2026-03-05 16:14:23.124933 | 
2026-03-05 16:14:23.125015 | TASK [add-build-sshkey : Install build public key as SSH key on all nodes]
2026-03-05 16:14:23.654050 | controller-1 | changed
2026-03-05 16:14:23.705551 | controller-3 | changed
2026-03-05 16:14:23.709930 | controller-2 | changed
2026-03-05 16:14:23.714142 | 
2026-03-05 16:14:23.714212 | TASK [add-build-sshkey : Remote setup ssh keys (windows)]
2026-03-05 16:14:23.741224 | controller-1 | skipping: Conditional result was False
2026-03-05 16:14:23.753153 | controller-2 | skipping: Conditional result was False
2026-03-05 16:14:23.753503 | controller-3 | skipping: Conditional result was False
2026-03-05 16:14:23.759610 | 
2026-03-05 16:14:23.759691 | TASK [remove-zuul-sshkey : Remove master key from local agent]
2026-03-05 16:14:24.204004 | controller-1 -> localhost | changed
2026-03-05 16:14:24.270785 | 
2026-03-05 16:14:24.270947 | TASK [add-build-sshkey : Add back temp key]
2026-03-05 16:14:24.546622 | controller-1 -> localhost | Identity added: /var/lib/zuul/builds/3f7a9e4c321043949fd99e3d8a4500aa/work/3f7a9e4c321043949fd99e3d8a4500aa_id_rsa (zuul-build-sshkey)
2026-03-05 16:14:24.546899 | controller-1 -> localhost | ok: Runtime: 0:00:00.013965
2026-03-05 16:14:24.552222 | 
2026-03-05 16:14:24.552286 | TASK [add-build-sshkey : Verify we can still SSH to all nodes]
2026-03-05 16:14:24.872935 | controller-3 | ok
2026-03-05 16:14:24.877491 | controller-1 | ok
2026-03-05 16:14:24.880759 | controller-2 | ok
2026-03-05 16:14:24.884511 | 
2026-03-05 16:14:24.884576 | TASK [add-build-sshkey : Verify we can still SSH to all nodes (windows)]
2026-03-05 16:14:24.919091 | controller-1 | skipping: Conditional result was False
2026-03-05 16:14:24.931244 | controller-2 | skipping: Conditional result was False
2026-03-05 16:14:24.942221 | controller-3 | skipping: Conditional result was False
2026-03-05 16:14:24.961525 | 
2026-03-05 16:14:24.961610 | TASK [prepare-workspace : Start zuul_console daemon.]
2026-03-05 16:14:25.272615 | controller-3 | ok
2026-03-05 16:14:25.276300 | controller-1 | ok
2026-03-05 16:14:25.302663 | controller-2 | ok
2026-03-05 16:14:25.310801 | 
2026-03-05 16:14:25.310866 | TASK [prepare-workspace : Synchronize src repos to workspace directory.]
2026-03-05 16:14:26.317756 | controller-1 | Output suppressed because no_log was given
2026-03-05 16:14:26.375643 | controller-2 | Output suppressed because no_log was given
2026-03-05 16:14:26.389162 | controller-3 | Output suppressed because no_log was given
2026-03-05 16:14:26.406133 | 
2026-03-05 16:14:26.406193 | LOOP [ensure-output-dirs : Empty Zuul Output directories by removing them]
2026-03-05 16:14:26.608461 | controller-1 | ok: "logs"
2026-03-05 16:14:26.608770 | controller-1 | ok: All items complete
2026-03-05 16:14:26.608816 | 
2026-03-05 16:14:26.619642 | controller-2 | ok: "logs"
2026-03-05 16:14:26.636942 | controller-3 | ok: "logs"
2026-03-05 16:14:26.815691 | controller-3 | ok: "artifacts"
2026-03-05 16:14:26.821979 | controller-1 | ok: "artifacts"
2026-03-05 16:14:26.825976 | controller-2 | ok: "artifacts"
2026-03-05 16:14:27.002237 | controller-1 | ok: "docs"
2026-03-05 16:14:27.007272 | controller-3 | ok: "docs"
2026-03-05 16:14:27.011779 | controller-2 | ok: "docs"
2026-03-05 16:14:27.022100 | 
2026-03-05 16:14:27.022269 | LOOP [ensure-output-dirs : Ensure Zuul Output directories exist]
2026-03-05 16:14:27.244796 | controller-1 | changed: "logs"
2026-03-05 16:14:27.245142 | controller-1 | ok: All items complete
2026-03-05 16:14:27.245180 | 
2026-03-05 16:14:27.249481 | controller-2 | changed: "logs"
2026-03-05 16:14:27.275071 | controller-3 | changed: "logs"
2026-03-05 16:14:27.413992 | controller-1 | changed: "artifacts"
2026-03-05 16:14:27.458361 | controller-3 | changed: "artifacts"
2026-03-05 16:14:27.461837 | controller-2 | changed: "artifacts"
2026-03-05 16:14:27.601976 | controller-1 | changed: "docs"
2026-03-05 16:14:27.643750 | controller-3 | changed: "docs"
2026-03-05 16:14:27.653884 | controller-2 | changed: "docs"
2026-03-05 16:14:27.689343 | 
2026-03-05 16:14:27.689432 | PLAY RECAP
2026-03-05 16:14:27.689503 | controller-1 | ok: 15 changed: 8 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0
2026-03-05 16:14:27.689546 | controller-2 | ok: 11 changed: 5 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0
2026-03-05 16:14:27.689581 | controller-3 | ok: 11 changed: 5 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0
2026-03-05 16:14:27.689614 | localhost | ok: 6 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0
2026-03-05 16:14:27.689639 | 
2026-03-05 16:14:27.901434 | PRE-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main]
2026-03-05 16:14:27.907091 | PRE-RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main]
2026-03-05 16:14:28.544052 | 
2026-03-05 16:14:28.544218 | PLAY [all]
2026-03-05 16:14:28.558087 | 
2026-03-05 16:14:28.558187 | TASK [setup-uv : Extract archive]
2026-03-05 16:14:30.890173 | controller-1 | changed
2026-03-05 16:14:30.896387 | controller-2 | changed
2026-03-05 16:14:30.975486 | controller-3 | changed
2026-03-05 16:14:30.987130 | 
2026-03-05 16:14:30.989089 | TASK [setup-uv : Print version]
2026-03-05 16:14:31.412875 | controller-3 | uv 0.8.13
2026-03-05 16:14:30.480639 | controller-1 | uv 0.8.13
2026-03-05 16:14:31.450225 | controller-2 | uv 0.8.13
2026-03-05 16:14:31.551523 | controller-1 | ok: Runtime: 0:00:00.012285
2026-03-05 16:14:31.563776 | controller-3 | ok: Runtime: 0:00:00.014826
2026-03-05 16:14:31.573203 | controller-2 | ok: Runtime: 0:00:00.012115
2026-03-05 16:14:31.587247 | 
2026-03-05 16:14:31.587314 | PLAY RECAP
2026-03-05 16:14:31.587367 | controller-1 | ok: 2 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0
2026-03-05 16:14:31.587396 | controller-2 | ok: 2 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0
2026-03-05 16:14:31.587423 | controller-3 | ok: 2 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0
2026-03-05 16:14:31.587442 | 
2026-03-05 16:14:31.737257 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main]
2026-03-05 16:14:31.744008 | RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/run.yaml@main]
2026-03-05 16:14:32.408742 | 
2026-03-05 16:14:32.408884 | PLAY [all]
2026-03-05 16:14:32.419977 | 
2026-03-05 16:14:32.420049 | TASK [Copy inventory file for Zuul]
2026-03-05 16:14:33.353234 | controller-1 | changed
2026-03-05 16:14:33.358696 | 
2026-03-05 16:14:33.358775 | TASK [Switch "ansible_host" to private IP]
2026-03-05 16:14:33.729069 | controller-1 | changed: 3 replacements made
2026-03-05 16:14:33.737315 | 
2026-03-05 16:14:33.737386 | TASK [Run Molecule scenario]
2026-03-05 16:14:33.311950 | controller-1 | Using CPython 3.13.5 interpreter at: /usr/bin/python3
2026-03-05 16:14:33.312078 | controller-1 | Creating virtual environment at: .venv
2026-03-05 16:14:34.516398 | controller-1 | Downloading pygments (1.2MiB)
2026-03-05 16:14:34.525817 | controller-1 | Downloading cryptography (4.3MiB)
2026-03-05 16:14:34.526217 | controller-1 | Downloading ansible-core (2.3MiB)
2026-03-05 16:14:34.697393 | controller-1 |  Downloading cryptography
2026-03-05 16:14:34.753565 | controller-1 |  Downloading pygments
2026-03-05 16:14:34.934341 | controller-1 |  Downloading ansible-core
2026-03-05 16:14:34.982923 | controller-1 | Installed 36 packages in 47ms
2026-03-05 16:14:35.645522 | controller-1 | INFO     Collection 'vexxhost.kubernetes' detected.
2026-03-05 16:14:35.645603 | controller-1 | INFO     Scenarios will be used from 'extensions/molecule'
2026-03-05 16:14:36.505484 | controller-1 | CRITICAL 'extensions/molecule/default/molecule.yml' glob failed.  Exiting.
2026-03-05 16:14:36.505536 | controller-1 | INFO     default scenario not found, disabling shared state.
2026-03-05 16:14:36.505673 | controller-1 | INFO     [cluster-upgrade > discovery] scenario test matrix: dependency, cleanup, destroy, syntax, create, prepare, converge, idempotence, side_effect, verify, cleanup, destroy
2026-03-05 16:14:36.505752 | controller-1 | INFO     [cluster-upgrade > prerun] Performing prerun with role_name_check=0...
2026-03-05 16:15:21.880295 | controller-1 | INFO     [cluster-upgrade > dependency] Executing
2026-03-05 16:15:21.880412 | controller-1 | WARNING  [cluster-upgrade > dependency] Missing roles requirements file: requirements.yml
2026-03-05 16:15:21.880616 | controller-1 | WARNING  [cluster-upgrade > dependency] Missing collections requirements file: collections.yml
2026-03-05 16:15:21.880739 | controller-1 | WARNING  [cluster-upgrade > dependency] Executed: 2 missing (Remove from test_sequence to suppress)
2026-03-05 16:15:21.886139 | controller-1 | INFO     [cluster-upgrade > cleanup] Executing
2026-03-05 16:15:21.886431 | controller-1 | WARNING  [cluster-upgrade > cleanup] Executed: Missing playbook (Remove from test_sequence to suppress)
2026-03-05 16:15:21.891781 | controller-1 | INFO     [cluster-upgrade > destroy] Executing
2026-03-05 16:15:21.891807 | controller-1 | WARNING  [cluster-upgrade > destroy] Skipping, '--destroy=never' requested.
2026-03-05 16:15:21.891879 | controller-1 | INFO     [cluster-upgrade > destroy] Executed: Successful
2026-03-05 16:15:21.897150 | controller-1 | INFO     [cluster-upgrade > syntax] Executing
2026-03-05 16:15:23.170768 | controller-1 |
2026-03-05 16:15:23.171001 | controller-1 | playbook: /home/zuul/src/github.com/vexxhost/ansible-collection-kubernetes/extensions/molecule/cluster-upgrade/converge.yml
2026-03-05 16:15:23.287984 | controller-1 | INFO     [cluster-upgrade > syntax] Executed: Successful
2026-03-05 16:15:23.300192 | controller-1 | INFO     [cluster-upgrade > create] Executing
2026-03-05 16:15:23.301605 | controller-1 | WARNING  [cluster-upgrade > create] Executed: Missing playbook (Remove from test_sequence to suppress)
2026-03-05 16:15:23.306887 | controller-1 | INFO     [cluster-upgrade > prepare] Executing
2026-03-05 16:15:24.575197 | controller-1 |
2026-03-05 16:15:24.575258 | controller-1 | PLAY [Prepare] *****************************************************************
2026-03-05 16:15:24.575272 | controller-1 |
2026-03-05 16:15:24.575442 | controller-1 | TASK [Gathering Facts] *********************************************************
2026-03-05 16:15:24.575515 | controller-1 | Thursday 05 March 2026  16:15:24 +0000 (0:00:00.029)       0:00:00.029 ********
2026-03-05 16:15:25.816911 | controller-1 | [WARNING]: Host 'controller-1' is using the discovered Python interpreter at '/usr/bin/python3.13', but future installation of another Python interpreter could cause a different interpreter to be discovered. See https://docs.ansible.com/ansible-core/2.20/reference_appendices/interpreter_discovery.html for more information.
2026-03-05 16:15:25.853508 | controller-1 | [WARNING]: Host 'controller-2' is using the discovered Python interpreter at '/usr/bin/python3.13', but future installation of another Python interpreter could cause a different interpreter to be discovered. See https://docs.ansible.com/ansible-core/2.20/reference_appendices/interpreter_discovery.html for more information.
2026-03-05 16:15:26.811984 | controller-1 | [WARNING]: Host 'controller-3' is using the discovered Python interpreter at '/usr/bin/python3.13', but future installation of another Python interpreter could cause a different interpreter to be discovered. See https://docs.ansible.com/ansible-core/2.20/reference_appendices/interpreter_discovery.html for more information.
2026-03-05 16:15:26.819715 | controller-1 | ok: [controller-1]
2026-03-05 16:15:26.819965 | controller-1 | ok: [controller-2]
2026-03-05 16:15:26.820157 | controller-1 | ok: [controller-3]
2026-03-05 16:15:26.820341 | controller-1 |
2026-03-05 16:15:26.820573 | controller-1 | TASK [Install Debian packages] *************************************************
2026-03-05 16:15:26.820772 | controller-1 | Thursday 05 March 2026  16:15:26 +0000 (0:00:02.244)       0:00:02.274 ********
2026-03-05 16:15:32.130585 | controller-1 | ok: [controller-3]
2026-03-05 16:15:32.130746 | controller-1 | ok: [controller-1]
2026-03-05 16:15:32.130943 | controller-1 | ok: [controller-2]
2026-03-05 16:15:32.131132 | controller-1 |
2026-03-05 16:15:32.131372 | controller-1 | PLAY [Configure Kubernetes VIP] ************************************************
2026-03-05 16:15:32.131624 | controller-1 |
2026-03-05 16:15:32.131823 | controller-1 | TASK [Gathering Facts] *********************************************************
2026-03-05 16:15:32.132009 | controller-1 | Thursday 05 March 2026  16:15:32 +0000 (0:00:05.311)       0:00:07.585 ********
2026-03-05 16:15:33.003461 | controller-1 | ok: [controller-1]
2026-03-05 16:15:33.003510 | controller-1 | ok: [controller-3]
2026-03-05 16:15:33.003519 | controller-1 | ok: [controller-2]
2026-03-05 16:15:33.003527 | controller-1 |
2026-03-05 16:15:33.003538 | controller-1 | TASK [vexxhost.containers.directory : Create directory (/etc/kubernetes/manifests)] ***
2026-03-05 16:15:33.003547 | controller-1 | Thursday 05 March 2026  16:15:32 +0000 (0:00:00.870)       0:00:08.456 ********
2026-03-05 16:15:33.446630 | controller-1 | changed: [controller-1]
2026-03-05 16:15:33.446769 | controller-1 | changed: [controller-3]
2026-03-05 16:15:33.446953 | controller-1 | changed: [controller-2]
2026-03-05 16:15:33.447125 | controller-1 |
2026-03-05 16:15:33.447355 | controller-1 | TASK [vexxhost.kubernetes.kube_vip : Uninstall legacy HA stack] ****************
2026-03-05 16:15:33.447585 | controller-1 | Thursday 05 March 2026  16:15:33 +0000 (0:00:00.445)       0:00:08.901 ********
2026-03-05 16:15:34.894742 | controller-1 | ok: [controller-1] => (item=/etc/keepalived/keepalived.conf)
2026-03-05 16:15:34.894768 | controller-1 | ok: [controller-3] => (item=/etc/keepalived/keepalived.conf)
2026-03-05 16:15:34.894771 | controller-1 | ok: [controller-2] => (item=/etc/keepalived/keepalived.conf)
2026-03-05 16:15:34.894775 | controller-1 | ok: [controller-1] => (item=/etc/keepalived/check_apiserver.sh)
2026-03-05 16:15:34.894778 | controller-1 | ok: [controller-3] => (item=/etc/keepalived/check_apiserver.sh)
2026-03-05 16:15:34.894781 | controller-1 | ok: [controller-2] => (item=/etc/keepalived/check_apiserver.sh)
2026-03-05 16:15:34.894785 | controller-1 | ok: [controller-1] => (item=/etc/kubernetes/manifests/keepalived.yaml)
2026-03-05 16:15:34.894789 | controller-1 | ok: [controller-3] => (item=/etc/kubernetes/manifests/keepalived.yaml)
2026-03-05 16:15:34.894792 | controller-1 | ok: [controller-2] => (item=/etc/kubernetes/manifests/keepalived.yaml)
2026-03-05 16:15:34.894795 | controller-1 | ok: [controller-1] => (item=/etc/haproxy/haproxy.cfg)
2026-03-05 16:15:34.894798 | controller-1 | ok: [controller-3] => (item=/etc/haproxy/haproxy.cfg)
2026-03-05 16:15:34.894801 | controller-1 | ok: [controller-2] => (item=/etc/haproxy/haproxy.cfg)
2026-03-05 16:15:34.894804 | controller-1 | ok: [controller-1] => (item=/etc/kubernetes/manifests/haproxy.yaml)
2026-03-05 16:15:34.894856 | controller-1 | ok: [controller-3] => (item=/etc/kubernetes/manifests/haproxy.yaml)
2026-03-05 16:15:34.895061 | controller-1 | ok: [controller-2] => (item=/etc/kubernetes/manifests/haproxy.yaml)
2026-03-05 16:15:34.895239 | controller-1 |
2026-03-05 16:15:34.895521 | controller-1 | TASK [vexxhost.kubernetes.kube_vip : Switch API server to run on port 6443] ****
2026-03-05 16:15:34.895718 | controller-1 | Thursday 05 March 2026  16:15:34 +0000 (0:00:01.445)       0:00:10.346 ********
2026-03-05 16:15:35.261747 | controller-1 | [ERROR]: Task failed: Module failed: Path /etc/kubernetes/manifests/kube-apiserver.yaml does not exist !
2026-03-05 16:15:35.261946 | controller-1 | Origin: /home/zuul/.ansible/collections/ansible_collections/vexxhost/kubernetes/roles/kube_vip/tasks/main.yml:32:3
2026-03-05 16:15:35.262128 | controller-1 |
2026-03-05 16:15:35.262334 | controller-1 | 30     - Restart "kubelet" service
2026-03-05 16:15:35.262512 | controller-1 | 31
2026-03-05 16:15:35.262705 | controller-1 | 32 - name: Switch API server to run on port 6443
2026-03-05 16:15:35.262887 | controller-1 |      ^ column 3
2026-03-05 16:15:35.263061 | controller-1 |
2026-03-05 16:15:35.263431 | controller-1 | failed: [controller-1] (item=/etc/kubernetes/manifests/kube-apiserver.yaml) => {"ansible_loop_var": "item", "changed": false, "item": "/etc/kubernetes/manifests/kube-apiserver.yaml", "msg": "Path /etc/kubernetes/manifests/kube-apiserver.yaml does not exist !", "rc": 257}
2026-03-05 16:15:35.270720 | controller-1 | failed: [controller-3] (item=/etc/kubernetes/manifests/kube-apiserver.yaml) => {"ansible_loop_var": "item", "changed": false, "item": "/etc/kubernetes/manifests/kube-apiserver.yaml", "msg": "Path /etc/kubernetes/manifests/kube-apiserver.yaml does not exist !", "rc": 257}
2026-03-05 16:15:35.288326 | controller-1 | failed: [controller-2] (item=/etc/kubernetes/manifests/kube-apiserver.yaml) => {"ansible_loop_var": "item", "changed": false, "item": "/etc/kubernetes/manifests/kube-apiserver.yaml", "msg": "Path /etc/kubernetes/manifests/kube-apiserver.yaml does not exist !", "rc": 257}
2026-03-05 16:15:35.511248 | controller-1 | [ERROR]: Task failed: Module failed: Path /etc/kubernetes/controller-manager.conf does not exist !
2026-03-05 16:15:35.511605 | controller-1 | Origin: /home/zuul/.ansible/collections/ansible_collections/vexxhost/kubernetes/roles/kube_vip/tasks/main.yml:32:3
2026-03-05 16:15:35.511805 | controller-1 |
2026-03-05 16:15:35.512066 | controller-1 | 30     - Restart "kubelet" service
2026-03-05 16:15:35.512262 | controller-1 | 31
2026-03-05 16:15:35.512588 | controller-1 | 32 - name: Switch API server to run on port 6443
2026-03-05 16:15:35.512881 | controller-1 |      ^ column 3
2026-03-05 16:15:35.513079 | controller-1 |
2026-03-05 16:15:35.513346 | controller-1 | failed: [controller-1] (item=/etc/kubernetes/controller-manager.conf) => {"ansible_loop_var": "item", "changed": false, "item": "/etc/kubernetes/controller-manager.conf", "msg": "Path /etc/kubernetes/controller-manager.conf does not exist !", "rc": 257}
2026-03-05 16:15:35.537273 | controller-1 | failed: [controller-3] (item=/etc/kubernetes/controller-manager.conf) => {"ansible_loop_var": "item", "changed": false, "item": "/etc/kubernetes/controller-manager.conf", "msg": "Path /etc/kubernetes/controller-manager.conf does not exist !", "rc": 257}
2026-03-05 16:15:35.554336 | controller-1 | failed: [controller-2] (item=/etc/kubernetes/controller-manager.conf) => {"ansible_loop_var": "item", "changed": false, "item": "/etc/kubernetes/controller-manager.conf", "msg": "Path /etc/kubernetes/controller-manager.conf does not exist !", "rc": 257}
2026-03-05 16:15:35.762019 | controller-1 | [ERROR]: Task failed: Module failed: Path /etc/kubernetes/scheduler.conf does not exist !
2026-03-05 16:15:35.762231 | controller-1 | Origin: /home/zuul/.ansible/collections/ansible_collections/vexxhost/kubernetes/roles/kube_vip/tasks/main.yml:32:3
2026-03-05 16:15:35.762425 | controller-1 |
2026-03-05 16:15:35.762629 | controller-1 | 30     - Restart "kubelet" service
2026-03-05 16:15:35.762865 | controller-1 | 31
2026-03-05 16:15:35.763097 | controller-1 | 32 - name: Switch API server to run on port 6443
2026-03-05 16:15:35.763331 | controller-1 |      ^ column 3
2026-03-05 16:15:35.763532 | controller-1 |
2026-03-05 16:15:35.763838 | controller-1 | failed: [controller-1] (item=/etc/kubernetes/scheduler.conf) => {"ansible_loop_var": "item", "changed": false, "item": "/etc/kubernetes/scheduler.conf", "msg": "Path /etc/kubernetes/scheduler.conf does not exist !", "rc": 257}
2026-03-05 16:15:35.804423 | controller-1 | failed: [controller-3] (item=/etc/kubernetes/scheduler.conf) => {"ansible_loop_var": "item", "changed": false, "item": "/etc/kubernetes/scheduler.conf", "msg": "Path /etc/kubernetes/scheduler.conf does not exist !", "rc": 257}
2026-03-05 16:15:35.815322 | controller-1 | failed: [controller-2] (item=/etc/kubernetes/scheduler.conf) => {"ansible_loop_var": "item", "changed": false, "item": "/etc/kubernetes/scheduler.conf", "msg": "Path /etc/kubernetes/scheduler.conf does not exist !", "rc": 257}
2026-03-05 16:15:35.824301 | controller-1 | ...ignoring
2026-03-05 16:15:35.824536 | controller-1 | ...ignoring
2026-03-05 16:15:35.824715 | controller-1 | ...ignoring
2026-03-05 16:15:35.824980 | controller-1 |
2026-03-05 16:15:35.825230 | controller-1 | TASK [vexxhost.kubernetes.kube_vip : Check if super-admin.conf exists] *********
2026-03-05 16:15:35.825421 | controller-1 | Thursday 05 March 2026  16:15:35 +0000 (0:00:00.932)       0:00:11.279 ********
2026-03-05 16:15:36.238426 | controller-1 | ok: [controller-1]
2026-03-05 16:15:36.238603 | controller-1 | ok: [controller-3]
2026-03-05 16:15:36.238797 | controller-1 | ok: [controller-2]
2026-03-05 16:15:36.238995 | controller-1 |
2026-03-05 16:15:36.239227 | controller-1 | TASK [vexxhost.kubernetes.kube_vip : Check if kubeadm has already run] *********
2026-03-05 16:15:36.239489 | controller-1 | Thursday 05 March 2026  16:15:36 +0000 (0:00:00.413)       0:00:11.692 ********
2026-03-05 16:15:36.560096 | controller-1 | ok: [controller-1]
2026-03-05 16:15:36.560249 | controller-1 | ok: [controller-2]
2026-03-05 16:15:36.560439 | controller-1 | ok: [controller-3]
2026-03-05 16:15:36.560613 | controller-1 |
2026-03-05 16:15:36.560817 | controller-1 | TASK [vexxhost.kubernetes.kube_vip : Set fact with KUBECONFIG path] ************
2026-03-05 16:15:36.561011 | controller-1 | Thursday 05 March 2026  16:15:36 +0000 (0:00:00.321)       0:00:12.014 ********
2026-03-05 16:15:36.613841 | controller-1 | ok: [controller-1]
2026-03-05 16:15:36.614006 | controller-1 | ok: [controller-2]
2026-03-05 16:15:36.614181 | controller-1 | ok: [controller-3]
2026-03-05 16:15:36.614388 | controller-1 |
2026-03-05 16:15:36.614589 | controller-1 | TASK [vexxhost.kubernetes.kube_vip : Set fact with KUBECONFIG path (with super-admin.conf)] ***
2026-03-05 16:15:36.614775 | controller-1 | Thursday 05 March 2026  16:15:36 +0000 (0:00:00.053)       0:00:12.068 ********
2026-03-05 16:15:36.673358 | controller-1 | ok: [controller-1]
2026-03-05 16:15:36.673511 | controller-1 | skipping: [controller-2]
2026-03-05 16:15:36.673696 | controller-1 | skipping: [controller-3]
2026-03-05 16:15:36.673869 | controller-1 |
2026-03-05 16:15:36.674065 | controller-1 | TASK [vexxhost.kubernetes.kube_vip : Upload Kubernetes manifest] ***************
2026-03-05 16:15:36.674273 | controller-1 | Thursday 05 March 2026  16:15:36 +0000 (0:00:00.059)       0:00:12.128 ********
2026-03-05 16:15:37.611207 | controller-1 | ok: [controller-1]
2026-03-05 16:15:37.614242 | controller-1 | ok: [controller-2]
2026-03-05 16:15:37.614253 | controller-1 | ok: [controller-3]
2026-03-05 16:15:37.614257 | controller-1 |
2026-03-05 16:15:37.614261 | controller-1 | TASK [vexxhost.kubernetes.kube_vip : Ensure kube-vip configuration file] *******
2026-03-05 16:15:37.614265 | controller-1 | Thursday 05 March 2026  16:15:37 +0000 (0:00:00.937)       0:00:13.065 ********
2026-03-05 16:15:38.019483 | controller-1 | ok: [controller-1]
2026-03-05 16:15:38.019646 | controller-1 | ok: [controller-2]
2026-03-05 16:15:38.019830 | controller-1 | ok: [controller-3]
2026-03-05 16:15:38.019986 | controller-1 |
2026-03-05 16:15:38.020154 | controller-1 | TASK [vexxhost.kubernetes.kube_vip : Flush handlers] ***************************
2026-03-05 16:15:38.020296 | controller-1 | Thursday 05 March 2026  16:15:37 +0000 (0:00:00.340)       0:00:13.406 ********
2026-03-05 16:15:38.020435 | controller-1 |
2026-03-05 16:15:38.020590 | controller-1 | TASK [vexxhost.kubernetes.kube_vip : Flush handlers] ***************************
2026-03-05 16:15:38.020736 | controller-1 | Thursday 05 March 2026  16:15:37 +0000 (0:00:00.004)       0:00:13.410 ********
2026-03-05 16:15:38.020867 | controller-1 |
2026-03-05 16:15:38.021019 | controller-1 | TASK [vexxhost.kubernetes.kube_vip : Flush handlers] ***************************
2026-03-05 16:15:38.021171 | controller-1 | Thursday 05 March 2026  16:15:37 +0000 (0:00:00.002)       0:00:13.413 ********
2026-03-05 16:15:38.021319 | controller-1 |
2026-03-05 16:15:38.021492 | controller-1 | PLAY [Install Kubernetes] ******************************************************
2026-03-05 16:15:38.021650 | controller-1 |
2026-03-05 16:15:38.021844 | controller-1 | TASK [Gathering Facts] *********************************************************
2026-03-05 16:15:38.022015 | controller-1 | Thursday 05 March 2026  16:15:38 +0000 (0:00:00.061)       0:00:13.474 ********
2026-03-05 16:15:38.890761 | controller-1 | ok: [controller-2]
2026-03-05 16:15:38.890998 | controller-1 | ok: [controller-3]
2026-03-05 16:15:38.891162 | controller-1 | ok: [controller-1]
2026-03-05 16:15:38.891241 | controller-1 |
2026-03-05 16:15:38.894975 | controller-1 | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-03-05 16:15:38.895022 | controller-1 | Thursday 05 March 2026  16:15:38 +0000 (0:00:00.870)       0:00:14.344 ********
2026-03-05 16:15:39.335568 | controller-1 | ok: [controller-1]
2026-03-05 16:15:39.335766 | controller-1 | ok: [controller-2]
2026-03-05 16:15:39.335970 | controller-1 | ok: [controller-3]
2026-03-05 16:15:39.336155 | controller-1 |
2026-03-05 16:15:39.336367 | controller-1 | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-03-05 16:15:39.336567 | controller-1 | Thursday 05 March 2026  16:15:39 +0000 (0:00:00.445)       0:00:14.790 ********
2026-03-05 16:15:39.414309 | controller-1 | skipping: [controller-1]
2026-03-05 16:15:39.414496 | controller-1 | skipping: [controller-2]
2026-03-05 16:15:39.414687 | controller-1 | skipping: [controller-3]
2026-03-05 16:15:39.414862 | controller-1 |
2026-03-05 16:15:39.415068 | controller-1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] ***
2026-03-05 16:15:39.415269 | controller-1 | Thursday 05 March 2026  16:15:39 +0000 (0:00:00.079)       0:00:14.869 ********
2026-03-05 16:15:39.751173 | controller-1 | changed: [controller-1]
2026-03-05 16:15:39.751356 | controller-1 | changed: [controller-2]
2026-03-05 16:15:39.751554 | controller-1 | changed: [controller-3]
2026-03-05 16:15:39.751780 | controller-1 |
2026-03-05 16:15:39.751985 | controller-1 | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-03-05 16:15:39.752178 | controller-1 | Thursday 05 March 2026  16:15:39 +0000 (0:00:00.336)       0:00:15.206 ********
2026-03-05 16:15:39.849150 | controller-1 | ok: [controller-1] => {
2026-03-05 16:15:39.849339 | controller-1 |     "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.0/runc.amd64"
2026-03-05 16:15:39.849538 | controller-1 | }
2026-03-05 16:15:39.849745 | controller-1 | ok: [controller-2] => {
2026-03-05 16:15:39.849973 | controller-1 |     "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.0/runc.amd64"
2026-03-05 16:15:39.850153 | controller-1 | }
2026-03-05 16:15:39.850345 | controller-1 | ok: [controller-3] => {
2026-03-05 16:15:39.850605 | controller-1 |     "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.0/runc.amd64"
2026-03-05 16:15:39.850801 | controller-1 | }
2026-03-05 16:15:39.850985 | controller-1 |
2026-03-05 16:15:39.851196 | controller-1 | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-03-05 16:15:39.851543 | controller-1 | Thursday 05 March 2026  16:15:39 +0000 (0:00:00.097)       0:00:15.303 ********
2026-03-05 16:15:40.590052 | controller-1 | [WARNING]: Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually
2026-03-05 16:15:40.648166 | controller-1 | changed: [controller-2]
2026-03-05 16:15:40.648263 | controller-1 | changed: [controller-3]
2026-03-05 16:15:40.648371 | controller-1 | changed: [controller-1]
2026-03-05 16:15:40.648475 | controller-1 |
2026-03-05 16:15:40.648590 | controller-1 | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-03-05 16:15:40.648709 | controller-1 | Thursday 05 March 2026  16:15:40 +0000 (0:00:00.799)       0:00:16.102 ********
2026-03-05 16:15:40.735025 | controller-1 | skipping: [controller-1]
2026-03-05 16:15:40.735171 | controller-1 | skipping: [controller-2]
2026-03-05 16:15:40.735411 | controller-1 | skipping: [controller-3]
2026-03-05 16:15:40.735576 | controller-1 |
2026-03-05 16:15:40.735769 | controller-1 | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-03-05 16:15:40.735955 | controller-1 | Thursday 05 March 2026  16:15:40 +0000 (0:00:00.087)       0:00:16.189 ********
2026-03-05 16:15:41.084553 | controller-1 | ok: [controller-1]
2026-03-05 16:15:41.084753 | controller-1 | ok: [controller-3]
2026-03-05 16:15:41.085010 | controller-1 | ok: [controller-2]
2026-03-05 16:15:41.085187 | controller-1 |
2026-03-05 16:15:41.085379 | controller-1 | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-03-05 16:15:41.085564 | controller-1 | Thursday 05 March 2026  16:15:41 +0000 (0:00:00.349)       0:00:16.539 ********
2026-03-05 16:15:42.001841 | controller-1 | ok: [controller-1]
2026-03-05 16:15:42.001875 | controller-1 | ok: [controller-3]
2026-03-05 16:15:42.001883 | controller-1 | ok: [controller-2]
2026-03-05 16:15:42.001891 | controller-1 |
2026-03-05 16:15:42.001900 | controller-1 | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-03-05 16:15:42.001908 | controller-1 | Thursday 05 March 2026  16:15:41 +0000 (0:00:00.916)       0:00:17.455 ********
2026-03-05 16:15:42.102661 | controller-1 | ok: [controller-1] => {
2026-03-05 16:15:42.102703 | controller-1 |     "msg": "https://github.com/containerd/containerd/releases/download/v2.2.0/containerd-2.2.0-linux-amd64.tar.gz"
2026-03-05 16:15:42.102821 | controller-1 | }
2026-03-05 16:15:42.102911 | controller-1 | ok: [controller-2] => {
2026-03-05 16:15:42.103044 | controller-1 |     "msg": "https://github.com/containerd/containerd/releases/download/v2.2.0/containerd-2.2.0-linux-amd64.tar.gz"
2026-03-05 16:15:42.103122 | controller-1 | }
2026-03-05 16:15:42.103455 | controller-1 | ok: [controller-3] => {
2026-03-05 16:15:42.103495 | controller-1 |     "msg": "https://github.com/containerd/containerd/releases/download/v2.2.0/containerd-2.2.0-linux-amd64.tar.gz"
2026-03-05 16:15:42.103504 | controller-1 | }
2026-03-05 16:15:42.103519 | controller-1 |
2026-03-05 16:15:42.103692 | controller-1 | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-03-05 16:15:42.103709 | controller-1 | Thursday 05 March 2026  16:15:42 +0000 (0:00:00.101)       0:00:17.557 ********
2026-03-05 16:15:43.206981 | controller-1 | changed: [controller-1]
2026-03-05 16:15:43.207061 | controller-1 | changed: [controller-2]
2026-03-05 16:15:43.207187 | controller-1 | changed: [controller-3]
2026-03-05 16:15:43.207340 | controller-1 |
2026-03-05 16:15:43.207793 | controller-1 | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-03-05 16:15:43.207821 | controller-1 | Thursday 05 March 2026  16:15:43 +0000 (0:00:01.101)       0:00:18.659 ********
2026-03-05 16:15:47.223458 | controller-1 | changed: [controller-3]
2026-03-05 16:15:47.223504 | controller-1 | changed: [controller-2]
2026-03-05 16:15:47.227353 | controller-1 | changed: [controller-1]
2026-03-05 16:15:47.227380 | controller-1 |
2026-03-05 16:15:47.227385 | controller-1 | TASK [vexxhost.containers.containerd : Install SELinux packages] ***************
2026-03-05 16:15:47.227389 | controller-1 | Thursday 05 March 2026  16:15:47 +0000 (0:00:04.016)       0:00:22.675 ********
2026-03-05 16:15:47.258342 | controller-1 | skipping: [controller-1]
2026-03-05 16:15:47.286261 | controller-1 | skipping: [controller-2]
2026-03-05 16:15:47.286300 | controller-1 | skipping: [controller-3]
2026-03-05 16:15:47.286394 | controller-1 |
2026-03-05 16:15:47.286492 | controller-1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] ***
2026-03-05 16:15:47.286578 | controller-1 | Thursday 05 March 2026  16:15:47 +0000 (0:00:00.063)       0:00:22.738 ********
2026-03-05 16:15:47.345136 | controller-1 | skipping: [controller-1]
2026-03-05 16:15:47.345184 | controller-1 | skipping: [controller-2]
2026-03-05 16:15:47.345350 | controller-1 | skipping: [controller-3]
2026-03-05 16:15:47.345464 | controller-1 |
2026-03-05 16:15:47.345565 | controller-1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ********
2026-03-05 16:15:47.345662 | controller-1 | Thursday 05 March 2026  16:15:47 +0000 (0:00:00.060)       0:00:22.799 ********
2026-03-05 16:15:47.410477 | controller-1 | skipping: [controller-1]
2026-03-05 16:15:47.410531 | controller-1 | skipping: [controller-2]
2026-03-05 16:15:47.410614 | controller-1 | skipping: [controller-3]
2026-03-05 16:15:47.410697 | controller-1 |
2026-03-05 16:15:47.410790 | controller-1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] **************
2026-03-05 16:15:47.410881 | controller-1 | Thursday 05 March 2026  16:15:47 +0000 (0:00:00.063)       0:00:22.863 ********
2026-03-05 16:15:56.119165 | controller-1 | changed: [controller-1]
2026-03-05 16:15:56.119211 | controller-1 | changed: [controller-3]
2026-03-05 16:15:56.119336 | controller-1 | changed: [controller-2]
2026-03-05 16:15:56.119644 | controller-1 |
2026-03-05 16:15:56.119669 | controller-1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] ***
2026-03-05 16:15:56.119673 | controller-1 | Thursday 05 March 2026  16:15:56 +0000 (0:00:08.711)       0:00:31.574 ********
2026-03-05 16:15:56.863664 | controller-1 | changed: [controller-1]
2026-03-05 16:15:56.863705 | controller-1 | changed: [controller-2]
2026-03-05 16:15:56.863818 | controller-1 | changed: [controller-3]
2026-03-05 16:15:56.863897 | controller-1 |
2026-03-05 16:15:56.864095 | controller-1 | TASK [vexxhost.containers.containerd : Create folders for configuration] *******
2026-03-05 16:15:56.864121 | controller-1 | Thursday 05 March 2026  16:15:56 +0000 (0:00:00.742)       0:00:32.316 ********
2026-03-05 16:15:58.319872 | controller-1 | changed: [controller-1] => (item={'path': '/etc/containerd'})
2026-03-05 16:15:58.319927 | controller-1 | changed: [controller-2] => (item={'path': '/etc/containerd'})
2026-03-05 16:15:58.319936 | controller-1 | changed: [controller-3] => (item={'path': '/etc/containerd'})
2026-03-05 16:15:58.320070 | controller-1 | changed: [controller-1] => (item={'path': '/var/lib/containerd', 'mode': '0o700'})
2026-03-05 16:15:58.320174 | controller-1 | changed: [controller-3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'})
2026-03-05 16:15:58.320268 | controller-1 | changed: [controller-2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'})
2026-03-05 16:15:58.320350 | controller-1 | changed: [controller-1] => (item={'path': '/run/containerd', 'mode': '0o711'})
2026-03-05 16:15:58.320448 | controller-1 | changed: [controller-3] => (item={'path': '/run/containerd', 'mode': '0o711'})
2026-03-05 16:15:58.320527 | controller-1 | changed: [controller-2] => (item={'path': '/run/containerd', 'mode': '0o711'})
2026-03-05 16:15:58.320617 | controller-1 | changed: [controller-1] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'})
2026-03-05 16:15:58.320706 | controller-1 | changed: [controller-3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'})
2026-03-05 16:15:58.320801 | controller-1 | changed: [controller-2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'})
2026-03-05 16:15:58.320891 | controller-1 | changed: [controller-1] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'})
2026-03-05 16:15:58.320981 | controller-1 | changed: [controller-3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'})
2026-03-05 16:15:58.321072 | controller-1 | changed: [controller-2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'})
2026-03-05 16:15:58.321348 | controller-1 |
2026-03-05 16:15:58.321377 | controller-1 | TASK [vexxhost.containers.containerd : Create containerd config file] **********
2026-03-05 16:15:58.321382 | controller-1 | Thursday 05 March 2026  16:15:58 +0000 (0:00:01.455)       0:00:33.771 ********
2026-03-05 16:15:59.151830 | controller-1 | changed: [controller-1]
2026-03-05 16:15:59.151888 | controller-1 | changed: [controller-3]
2026-03-05 16:15:59.152044 | controller-1 | changed: [controller-2]
2026-03-05 16:15:59.152207 | controller-1 |
2026-03-05 16:15:59.152375 | controller-1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ********
2026-03-05 16:15:59.152515 | controller-1 | Thursday 05 March 2026  16:15:59 +0000 (0:00:00.806)       0:00:34.578 ********
2026-03-05 16:15:59.152678 | controller-1 |
2026-03-05 16:15:59.152819 | controller-1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ********
2026-03-05 16:15:59.152955 | controller-1 | Thursday 05 March 2026  16:15:59 +0000 (0:00:00.013)       0:00:34.592 ********
2026-03-05 16:15:59.153071 | controller-1 |
2026-03-05 16:15:59.153207 | controller-1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ********
2026-03-05 16:15:59.153359 | controller-1 | Thursday 05 March 2026  16:15:59 +0000 (0:00:00.005)       0:00:34.598 ********
2026-03-05 16:15:59.153470 | controller-1 |
2026-03-05 16:15:59.153613 | controller-1 | RUNNING HANDLER [vexxhost.containers.containerd : Restart containerd] **********
2026-03-05 16:15:59.153752 | controller-1 | Thursday 05 March 2026  16:15:59 +0000 (0:00:00.007)       0:00:34.605 ********
2026-03-05 16:16:00.159768 | controller-1 | changed: [controller-3]
2026-03-05 16:16:00.159856 | controller-1 | changed: [controller-1]
2026-03-05 16:16:00.159979 | controller-1 | changed: [controller-2]
2026-03-05 16:16:00.160090 | controller-1 |
2026-03-05 16:16:00.160354 | controller-1 | RUNNING HANDLER [vexxhost.kubernetes.kubelet : Reload systemd] *****************
2026-03-05 16:16:00.160382 | controller-1 | Thursday 05 March 2026  16:16:00 +0000 (0:00:01.006)       0:00:35.612 ********
2026-03-05 16:16:02.138753 | controller-1 | ok: [controller-1]
2026-03-05 16:16:02.138807 | controller-1 | ok: [controller-3]
2026-03-05 16:16:02.138944 | controller-1 | ok: [controller-2]
2026-03-05 16:16:02.139019 | controller-1 |
2026-03-05 16:16:02.139122 | controller-1 | TASK [vexxhost.containers.containerd : Enable and start service] ***************
2026-03-05 16:16:02.139189 | controller-1 | Thursday 05 March 2026  16:16:02 +0000 (0:00:01.980)       0:00:37.593 ********
2026-03-05 16:16:02.970759 | controller-1 | changed: [controller-1]
2026-03-05 16:16:02.970808 | controller-1 | changed: [controller-3]
2026-03-05 16:16:02.970881 | controller-1 | changed: [controller-2]
2026-03-05 16:16:02.970970 | controller-1 |
2026-03-05 16:16:02.971061 | controller-1 | TASK [vexxhost.kubernetes.kubernetes_upgrade_check : Retrieve the "kubeadm-config" ConfigMap] ***
2026-03-05 16:16:02.971129 | controller-1 | Thursday 05 March 2026  16:16:02 +0000 (0:00:00.829)       0:00:38.423 ********
2026-03-05 16:16:03.605749 | controller-1 | [ERROR]: Task failed: Module failed: Failed to import the required Python library (kubernetes) on np0000157878's Python /usr/bin/python3.13. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter
2026-03-05 16:16:03.605955 | controller-1 | Origin: /home/zuul/.ansible/collections/ansible_collections/vexxhost/kubernetes/roles/kubernetes_upgrade_check/tasks/main.yml:16:3
2026-03-05 16:16:03.606133 | controller-1 |
2026-03-05 16:16:03.606319 | controller-1 | 14
2026-03-05 16:16:03.606501 | controller-1 | 15 ---
2026-03-05 16:16:03.606695 | controller-1 | 16 - name: Retrieve the "kubeadm-config" ConfigMap
2026-03-05 16:16:03.606876 | controller-1 |      ^ column 3
2026-03-05 16:16:03.607049 | controller-1 |
2026-03-05 16:16:03.611595 | controller-1 | fatal: [controller-1]: FAILED! => {"changed": false, "msg": "Failed to import the required Python library (kubernetes) on np0000157878's Python /usr/bin/python3.13. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"}
2026-03-05 16:16:03.619076 | controller-1 | ...ignoring
2026-03-05 16:16:03.619182 | controller-1 |
2026-03-05 16:16:03.619349 | controller-1 | TASK [vexxhost.kubernetes.kubernetes_upgrade_check : Parse the ClusterConfiguration] ***
2026-03-05 16:16:03.619532 | controller-1 | Thursday 05 March 2026  16:16:03 +0000 (0:00:00.650)       0:00:39.073 ********
2026-03-05 16:16:03.663758 | controller-1 | skipping: [controller-1]
2026-03-05 16:16:03.663868 | controller-1 |
2026-03-05 16:16:03.664001 | controller-1 | TASK [vexxhost.kubernetes.kubernetes_upgrade_check : Retrieve the current Kubernetes version] ***
2026-03-05 16:16:03.664130 | controller-1 | Thursday 05 March 2026  16:16:03 +0000 (0:00:00.041)       0:00:39.115 ********
2026-03-05 16:16:03.703764 | controller-1 | skipping: [controller-1]
2026-03-05 16:16:03.703789 | controller-1 |
2026-03-05 16:16:03.703794 | controller-1 | TASK [vexxhost.kubernetes.kubernetes_upgrade_check : Extract major, minor, and patch versions] ***
2026-03-05 16:16:03.703798 | controller-1 | Thursday 05 March 2026  16:16:03 +0000 (0:00:00.042)       0:00:39.158 ********
2026-03-05 16:16:03.746622 | controller-1 | skipping: [controller-1]
2026-03-05 16:16:03.746680 | controller-1 |
2026-03-05 16:16:03.746783 | controller-1 | TASK [vexxhost.kubernetes.kubernetes_upgrade_check : Fail if we're jumping more than one minor version] ***
2026-03-05 16:16:03.746870 | controller-1 | Thursday 05 March 2026  16:16:03 +0000 (0:00:00.043)       0:00:39.201 ********
2026-03-05 16:16:03.807457 | controller-1 | skipping: [controller-1]
2026-03-05 16:16:03.807504 | controller-1 | skipping: [controller-2]
2026-03-05 16:16:03.807586 | controller-1 | skipping: [controller-3]
2026-03-05 16:16:03.807672 | controller-1 |
2026-03-05 16:16:03.807752 | controller-1 | TASK [vexxhost.kubernetes.kubernetes_upgrade_check : Set fact if we need to upgrade] ***
2026-03-05 16:16:03.807848 | controller-1 | Thursday 05 March 2026  16:16:03 +0000 (0:00:00.060)       0:00:39.262 ********
2026-03-05 16:16:03.882641 | controller-1 | skipping: [controller-1]
2026-03-05 16:16:03.882687 | controller-1 | skipping: [controller-2]
2026-03-05 16:16:03.882699 | controller-1 | skipping: [controller-3]
2026-03-05 16:16:03.882814 | controller-1 |
2026-03-05 16:16:03.882903 | controller-1 | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-03-05 16:16:03.882991 | controller-1 | Thursday 05 March 2026  16:16:03 +0000 (0:00:00.075)       0:00:39.337 ********
2026-03-05 16:16:04.251739 | controller-1 | ok: [controller-1]
2026-03-05 16:16:04.251797 | controller-1 | ok: [controller-3]
2026-03-05 16:16:04.251809 | controller-1 | ok: [controller-2]
2026-03-05 16:16:04.251934 | controller-1 |
2026-03-05 16:16:04.252010 | controller-1 | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-03-05 16:16:04.252099 | controller-1 | Thursday 05 March 2026  16:16:04 +0000 (0:00:00.367)       0:00:39.705 ********
2026-03-05 16:16:04.316483 | controller-1 | ok: [controller-1] => {
2026-03-05 16:16:04.316559 | controller-1 |     "msg": "https://dl.k8s.io/release/v1.33.4/bin/linux/amd64/kubeadm"
2026-03-05 16:16:04.316653 | controller-1 | }
2026-03-05 16:16:04.316752 | controller-1 | ok: [controller-2] => {
2026-03-05 16:16:04.316831 | controller-1 |     "msg": "https://dl.k8s.io/release/v1.33.4/bin/linux/amd64/kubeadm"
2026-03-05 16:16:04.316907 | controller-1 | }
2026-03-05 16:16:04.317004 | controller-1 | ok: [controller-3] => {
2026-03-05 16:16:04.317087 | controller-1 |     "msg": "https://dl.k8s.io/release/v1.33.4/bin/linux/amd64/kubeadm"
2026-03-05 16:16:04.317216 | controller-1 | }
2026-03-05 16:16:04.317294 | controller-1 |
2026-03-05 16:16:04.317384 | controller-1 | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-03-05 16:16:04.317472 | controller-1 | Thursday 05 March 2026  16:16:04 +0000 (0:00:00.065)       0:00:39.771 ********
2026-03-05 16:16:29.776400 | controller-1 | changed: [controller-3]
2026-03-05 16:16:29.776429 | controller-1 | changed: [controller-2]
2026-03-05 16:16:29.776435 | controller-1 | changed: [controller-1]
2026-03-05 16:16:29.776440 | controller-1 |
2026-03-05 16:16:29.776445 | controller-1 | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-03-05 16:16:29.776456 | controller-1 | Thursday 05 March 2026  16:16:29 +0000 (0:00:25.457)       0:01:05.229 ********
2026-03-05 16:16:29.866726 | controller-1 | skipping: [controller-1]
2026-03-05 16:16:29.868144 | controller-1 | skipping: [controller-2]
2026-03-05 16:16:29.868465 | controller-1 | skipping: [controller-3]
2026-03-05 16:16:29.868704 | controller-1 |
2026-03-05 16:16:29.868962 | controller-1 | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-03-05 16:16:29.869212 | controller-1 | Thursday 05 March 2026  16:16:29 +0000 (0:00:00.092)       0:01:05.321 ********
2026-03-05 16:16:30.231086 | controller-1 | ok: [controller-1]
2026-03-05 16:16:30.231145 | controller-1 | ok: [controller-2]
2026-03-05 16:16:30.231209 | controller-1 | ok: [controller-3]
2026-03-05 16:16:30.231284 | controller-1 |
2026-03-05 16:16:30.231526 | controller-1 | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-03-05 16:16:30.231552 | controller-1 | Thursday 05 March 2026  16:16:30 +0000 (0:00:00.361)       0:01:05.683 ********
2026-03-05 16:16:30.303747 | controller-1 | ok: [controller-1] => {
2026-03-05 16:16:30.303847 | controller-1 |     "msg": "https://dl.k8s.io/release/v1.33.4/bin/linux/amd64/kubectl"
2026-03-05 16:16:30.303969 | controller-1 | }
2026-03-05 16:16:30.304096 | controller-1 | ok: [controller-2] => {
2026-03-05 16:16:30.304213 | controller-1 |     "msg": "https://dl.k8s.io/release/v1.33.4/bin/linux/amd64/kubectl"
2026-03-05 16:16:30.304318 | controller-1 | }
2026-03-05 16:16:30.304445 | controller-1 | ok: [controller-3] => {
2026-03-05 16:16:30.304558 | controller-1 |     "msg": "https://dl.k8s.io/release/v1.33.4/bin/linux/amd64/kubectl"
2026-03-05 16:16:30.304669 | controller-1 | }
2026-03-05 16:16:30.304778 | controller-1 |
2026-03-05 16:16:30.304901 | controller-1 | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-03-05 16:16:30.305019 | controller-1 | Thursday 05 March 2026  16:16:30 +0000 (0:00:00.072)       0:01:05.755 ********
2026-03-05 16:16:32.072601 | controller-1 | changed: [controller-1]
2026-03-05 16:16:32.072649 | controller-1 | changed: [controller-3]
2026-03-05 16:16:32.072768 | controller-1 | changed: [controller-2]
2026-03-05 16:16:32.072883 | controller-1 |
2026-03-05 16:16:32.073017 | controller-1 | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-03-05 16:16:32.073158 | controller-1 | Thursday 05 March 2026  16:16:32 +0000 (0:00:01.771)       0:01:07.527 ********
2026-03-05 16:16:32.177744 | controller-1 | skipping: [controller-1]
2026-03-05 16:16:32.177802 | controller-1 | skipping: [controller-2]
2026-03-05 16:16:32.177811 | controller-1 | skipping: [controller-3]
2026-03-05 16:16:32.177883 | controller-1 |
2026-03-05 16:16:32.177982 | controller-1 | TASK [vexxhost.containers.containerd : Install SELinux packages] ***************
2026-03-05 16:16:32.178080 | controller-1 | Thursday 05 March 2026  16:16:32 +0000 (0:00:00.105)       0:01:07.632 ********
2026-03-05 16:16:32.239729 | controller-1 | skipping: [controller-1]
2026-03-05 16:16:32.239785 | controller-1 | skipping: [controller-2]
2026-03-05 16:16:32.239795 | controller-1 | skipping: [controller-3]
2026-03-05 16:16:32.239874 | controller-1 |
2026-03-05 16:16:32.239985 | controller-1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] ***
2026-03-05 16:16:32.240084 | controller-1 | Thursday 05 March 2026  16:16:32 +0000 (0:00:00.060)       0:01:07.693 ********
2026-03-05 16:16:32.297876 | controller-1 | skipping: [controller-1]
2026-03-05 16:16:32.297947 | controller-1 | skipping: [controller-2]
2026-03-05 16:16:32.298080 | controller-1 | skipping: [controller-3]
2026-03-05 16:16:32.298184 | controller-1 |
2026-03-05 16:16:32.298460 | controller-1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ********
2026-03-05 16:16:32.298493 | controller-1 | Thursday 05 March 2026  16:16:32 +0000 (0:00:00.059)       0:01:07.752 ********
2026-03-05 16:16:32.362899 | controller-1 | skipping: [controller-1]
2026-03-05 16:16:32.363013 | controller-1 | skipping: [controller-2]
2026-03-05 16:16:32.363137 | controller-1 | skipping: [controller-3]
2026-03-05 16:16:32.363281 | controller-1 |
2026-03-05 16:16:32.363597 | controller-1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] **************
2026-03-05 16:16:32.363626 | controller-1 | Thursday 05 March 2026  16:16:32 +0000 (0:00:00.064)       0:01:07.817 ********
2026-03-05 16:16:33.359660 | controller-1 | ok: [controller-1]
2026-03-05 16:16:33.359801 | controller-1 | ok: [controller-2]
2026-03-05 16:16:33.359913 | controller-1 | ok: [controller-3]
2026-03-05 16:16:33.360021 | controller-1 |
2026-03-05 16:16:33.360142 | controller-1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] ***
2026-03-05 16:16:33.360258 | controller-1 | Thursday 05 March 2026  16:16:33 +0000 (0:00:00.994)       0:01:08.811 ********
2026-03-05 16:16:34.075737 | controller-1 | ok: [controller-1]
2026-03-05 16:16:34.075789 | controller-1 | ok: [controller-2]
2026-03-05 16:16:34.075853 | controller-1 | ok: [controller-3]
2026-03-05 16:16:34.075943 | controller-1 |
2026-03-05 16:16:34.076034 | controller-1 | TASK [vexxhost.containers.containerd : Create folders for configuration] *******
2026-03-05 16:16:34.076123 | controller-1 | Thursday 05 March 2026  16:16:34 +0000 (0:00:00.716)       0:01:09.528 ********
2026-03-05 16:16:35.510119 | controller-1 | ok: [controller-1] => (item={'path': '/etc/containerd'})
2026-03-05 16:16:35.510177 | controller-1 | ok: [controller-2] => (item={'path': '/etc/containerd'})
2026-03-05 16:16:35.511425 | controller-1 | ok: [controller-3] => (item={'path': '/etc/containerd'})
2026-03-05 16:16:35.511468 | controller-1 | ok: [controller-1] => (item={'path': '/var/lib/containerd', 'mode': '0o700'})
2026-03-05 16:16:35.511477 | controller-1 | ok: [controller-2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'})
2026-03-05 16:16:35.511485 | controller-1 | ok: [controller-3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'})
2026-03-05 16:16:35.511492 | controller-1 | ok: [controller-1] => (item={'path': '/run/containerd', 'mode': '0o711'})
2026-03-05 16:16:35.511499 | controller-1 | ok: [controller-2] => (item={'path': '/run/containerd', 'mode': '0o711'})
2026-03-05 16:16:35.511506 | controller-1 | ok: [controller-3] => (item={'path': '/run/containerd', 'mode': '0o711'})
2026-03-05 16:16:35.511513 | controller-1 | ok: [controller-1] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'})
2026-03-05 16:16:35.511520 | controller-1 | ok: [controller-2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'})
2026-03-05 16:16:35.511547 | controller-1 | ok: [controller-3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'})
2026-03-05 16:16:35.512359 | controller-1 | ok: [controller-1] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'})
2026-03-05 16:16:35.512393 | controller-1 | ok: [controller-3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'})
2026-03-05 16:16:35.512399 | controller-1 | ok: [controller-2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'})
2026-03-05 16:16:35.512404 | controller-1 |
2026-03-05 16:16:35.512409 | controller-1 | TASK [vexxhost.containers.containerd : Create containerd config file] **********
2026-03-05 16:16:35.512414 | controller-1 | Thursday 05 March 2026  16:16:35 +0000 (0:00:01.435)       0:01:10.964 ********
2026-03-05 16:16:36.327703 | controller-1 | ok: [controller-1]
2026-03-05 16:16:36.327753 | controller-1 | ok: [controller-3]
2026-03-05 16:16:36.327814 | controller-1 | ok: [controller-2]
2026-03-05 16:16:36.327895 | controller-1 |
2026-03-05 16:16:36.327987 | controller-1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ********
2026-03-05 16:16:36.328071 | controller-1 | Thursday 05 March 2026  16:16:36 +0000 (0:00:00.800)       0:01:11.764 ********
2026-03-05 16:16:36.328142 | controller-1 |
2026-03-05 16:16:36.328242 | controller-1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ********
2026-03-05 16:16:36.328320 | controller-1 | Thursday 05 March 2026  16:16:36 +0000 (0:00:00.006)       0:01:11.770 ********
2026-03-05 16:16:36.328399 | controller-1 |
2026-03-05 16:16:36.328486 | controller-1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ********
2026-03-05 16:16:36.328577 | controller-1 | Thursday 05 March 2026  16:16:36 +0000 (0:00:00.004)       0:01:11.775 ********
2026-03-05 16:16:36.328652 | controller-1 |
2026-03-05 16:16:36.328991 | controller-1 | TASK [vexxhost.containers.containerd : Enable and start service] ***************
2026-03-05 16:16:36.329017 | controller-1 | Thursday 05 March 2026  16:16:36 +0000 (0:00:00.004)       0:01:11.779 ********
2026-03-05 16:16:36.868552 | controller-1 | ok: [controller-1]
2026-03-05 16:16:36.868587 | controller-1 | ok: [controller-2]
2026-03-05 16:16:36.868597 | controller-1 | ok: [controller-3]
2026-03-05 16:16:36.868606 | controller-1 |
2026-03-05 16:16:36.868616 | controller-1 | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-03-05 16:16:36.868624 | controller-1 | Thursday 05 March 2026  16:16:36 +0000 (0:00:00.539)       0:01:12.319 ********
2026-03-05 16:16:37.196560 | controller-1 | ok: [controller-1]
2026-03-05 16:16:37.196606 | controller-1 | ok: [controller-3]
2026-03-05 16:16:37.196707 | controller-1 | ok: [controller-2]
2026-03-05 16:16:37.196781 | controller-1 |
2026-03-05 16:16:37.196887 | controller-1 | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-03-05 16:16:37.196968 | controller-1 | Thursday 05 March 2026  16:16:37 +0000 (0:00:00.331)       0:01:12.651 ********
2026-03-05 16:16:37.288881 | controller-1 | ok: [controller-1] => {
2026-03-05 16:16:37.288929 | controller-1 |     "msg": "https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.34.0/crictl-v1.34.0-linux-amd64.tar.gz"
2026-03-05 16:16:37.289049 | controller-1 | }
2026-03-05 16:16:37.289173 | controller-1 | ok: [controller-2] => {
2026-03-05 16:16:37.289303 | controller-1 |     "msg": "https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.34.0/crictl-v1.34.0-linux-amd64.tar.gz"
2026-03-05 16:16:37.289420 | controller-1 | }
2026-03-05 16:16:37.289533 | controller-1 | ok: [controller-3] => {
2026-03-05 16:16:37.289674 | controller-1 |     "msg": "https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.34.0/crictl-v1.34.0-linux-amd64.tar.gz"
2026-03-05 16:16:37.289779 | controller-1 | }
2026-03-05 16:16:37.289890 | controller-1 |
2026-03-05 16:16:37.290180 | controller-1 | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-03-05 16:16:37.290207 | controller-1 | Thursday 05 March 2026  16:16:37 +0000 (0:00:00.091)       0:01:12.743 ********
2026-03-05 16:16:38.031757 | controller-1 | changed: [controller-2]
2026-03-05 16:16:38.031835 | controller-1 | changed: [controller-3]
2026-03-05 16:16:38.031963 | controller-1 | changed: [controller-1]
2026-03-05 16:16:38.032334 | controller-1 |
2026-03-05 16:16:38.032362 | controller-1 | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-03-05 16:16:38.032368 | controller-1 | Thursday 05 March 2026  16:16:38 +0000 (0:00:00.739)       0:01:13.483 ********
2026-03-05 16:16:40.746749 | controller-1 | changed: [controller-3]
2026-03-05 16:16:40.746796 | controller-1 | changed: [controller-1]
2026-03-05 16:16:40.746860 | controller-1 | changed: [controller-2]
2026-03-05 16:16:40.747121 | controller-1 |
2026-03-05 16:16:40.747144 | controller-1 | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-03-05 16:16:40.747148 | controller-1 | Thursday 05 March 2026  16:16:40 +0000 (0:00:02.718)       0:01:16.201 ********
2026-03-05 16:16:40.843523 | controller-1 | ok: [controller-1] => {
2026-03-05 16:16:40.843593 | controller-1 |     "msg": "https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.34.0/critest-v1.34.0-linux-amd64.tar.gz"
2026-03-05 16:16:40.843688 | controller-1 | }
2026-03-05 16:16:40.843788 | controller-1 | ok: [controller-2] => {
2026-03-05 16:16:40.843884 | controller-1 |     "msg": "https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.34.0/critest-v1.34.0-linux-amd64.tar.gz"
2026-03-05 16:16:40.843959 | controller-1 | }
2026-03-05 16:16:40.844048 | controller-1 | ok: [controller-3] => {
2026-03-05 16:16:40.844153 | controller-1 |     "msg": "https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.34.0/critest-v1.34.0-linux-amd64.tar.gz"
2026-03-05 16:16:40.844221 | controller-1 | }
2026-03-05 16:16:40.844297 | controller-1 |
2026-03-05 16:16:40.844386 | controller-1 | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-03-05 16:16:40.844471 | controller-1 | Thursday 05 March 2026  16:16:40 +0000 (0:00:00.096)       0:01:16.298 ********
2026-03-05 16:16:41.582739 | controller-1 | changed: [controller-2]
2026-03-05 16:16:41.583409 | controller-1 | changed: [controller-3]
2026-03-05 16:16:41.583450 | controller-1 | changed: [controller-1]
2026-03-05 16:16:41.583459 | controller-1 |
2026-03-05 16:16:41.583469 | controller-1 | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-03-05 16:16:41.583576 | controller-1 | Thursday 05 March 2026  16:16:41 +0000 (0:00:00.736)       0:01:17.034 ********
2026-03-05 16:16:43.271768 | controller-1 | changed: [controller-1]
2026-03-05 16:16:43.271820 | controller-1 | changed: [controller-2]
2026-03-05 16:16:43.271831 | controller-1 | changed: [controller-3]
2026-03-05 16:16:43.271940 | controller-1 |
2026-03-05 16:16:43.272024 | controller-1 | TASK [vexxhost.containers.cri_tools : Create crictl config] ********************
2026-03-05 16:16:43.272129 | controller-1 | Thursday 05 March 2026  16:16:43 +0000 (0:00:01.690)       0:01:18.725 ********
2026-03-05 16:16:43.995269 | controller-1 | changed: [controller-1]
2026-03-05 16:16:43.995339 | controller-1 | changed: [controller-3]
2026-03-05 16:16:43.995357 | controller-1 | changed: [controller-2]
2026-03-05 16:16:43.995474 | controller-1 |
2026-03-05 16:16:43.995576 | controller-1 | TASK [vexxhost.containers.directory : Create directory (/opt/cni/bin)] *********
2026-03-05 16:16:43.995646 | controller-1 | Thursday 05 March 2026  16:16:43 +0000 (0:00:00.725)       0:01:19.450 ********
2026-03-05 16:16:44.333433 | controller-1 | changed: [controller-1]
2026-03-05 16:16:44.333583 | controller-1 | changed: [controller-2]
2026-03-05 16:16:44.333823 | controller-1 | changed: [controller-3]
2026-03-05 16:16:44.333953 | controller-1 |
2026-03-05 16:16:44.334207 | controller-1 | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-03-05 16:16:44.334344 | controller-1 | Thursday 05 March 2026  16:16:44 +0000 (0:00:00.337)       0:01:19.788 ********
2026-03-05 16:16:44.658576 | controller-1 | ok: [controller-1]
2026-03-05 16:16:44.658718 | controller-1 | ok: [controller-3]
2026-03-05 16:16:44.658904 | controller-1 | ok: [controller-2]
2026-03-05 16:16:44.659092 | controller-1 |
2026-03-05 16:16:44.659270 | controller-1 | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-03-05 16:16:44.659547 | controller-1 | Thursday 05 March 2026  16:16:44 +0000 (0:00:00.325)       0:01:20.113 ********
2026-03-05 16:16:44.749243 | controller-1 | ok: [controller-1] => {
2026-03-05 16:16:44.749328 | controller-1 |     "msg": "https://github.com/containernetworking/plugins/releases/download/v1.8.0/cni-plugins-linux-amd64-v1.8.0.tgz"
2026-03-05 16:16:44.749405 | controller-1 | }
2026-03-05 16:16:44.749534 | controller-1 | ok: [controller-2] => {
2026-03-05 16:16:44.749632 | controller-1 |     "msg": "https://github.com/containernetworking/plugins/releases/download/v1.8.0/cni-plugins-linux-amd64-v1.8.0.tgz"
2026-03-05 16:16:44.749732 | controller-1 | }
2026-03-05 16:16:44.749861 | controller-1 | ok: [controller-3] => {
2026-03-05 16:16:44.749988 | controller-1 |     "msg": "https://github.com/containernetworking/plugins/releases/download/v1.8.0/cni-plugins-linux-amd64-v1.8.0.tgz"
2026-03-05 16:16:44.750105 | controller-1 | }
2026-03-05 16:16:44.750222 | controller-1 |
2026-03-05 16:16:44.750352 | controller-1 | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-03-05 16:16:44.750476 | controller-1 | Thursday 05 March 2026  16:16:44 +0000 (0:00:00.090)       0:01:20.204 ********
2026-03-05 16:16:45.904016 | controller-1 | changed: [controller-1]
2026-03-05 16:16:45.904055 | controller-1 | changed: [controller-3]
2026-03-05 16:16:45.904065 | controller-1 | changed: [controller-2]
2026-03-05 16:16:45.904073 | controller-1 |
2026-03-05 16:16:45.904081 | controller-1 | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-03-05 16:16:45.904089 | controller-1 | Thursday 05 March 2026  16:16:45 +0000 (0:00:01.153)       0:01:21.357 ********
2026-03-05 16:16:49.846526 | controller-1 | changed: [controller-1]
2026-03-05 16:16:49.846605 | controller-1 | changed: [controller-2]
2026-03-05 16:16:49.846618 | controller-1 | changed: [controller-3]
2026-03-05 16:16:49.846724 | controller-1 |
2026-03-05 16:16:49.846877 | controller-1 | TASK [vexxhost.containers.cni_plugins : Gather variables for each operating system] ***
2026-03-05 16:16:49.846950 | controller-1 | Thursday 05 March 2026  16:16:49 +0000 (0:00:03.943)       0:01:25.301 ********
2026-03-05 16:16:49.928381 | controller-1 | ok: [controller-1] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/containers/roles/cni_plugins/vars/debian.yml)
2026-03-05 16:16:49.928408 | controller-1 | ok: [controller-2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/containers/roles/cni_plugins/vars/debian.yml)
2026-03-05 16:16:49.928414 | controller-1 | ok: [controller-3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/containers/roles/cni_plugins/vars/debian.yml)
2026-03-05 16:16:49.928419 | controller-1 |
2026-03-05 16:16:49.928425 | controller-1 | TASK [vexxhost.containers.cni_plugins : Install additional packages] ***********
2026-03-05 16:16:49.928430 | controller-1 | Thursday 05 March 2026  16:16:49 +0000 (0:00:00.079)       0:01:25.381 ********
2026-03-05 16:16:50.927538 | controller-1 | ok: [controller-1]
2026-03-05 16:16:50.927583 | controller-1 | ok: [controller-2]
2026-03-05 16:16:50.931362 | controller-1 | ok: [controller-3]
2026-03-05 16:16:50.931388 | controller-1 |
2026-03-05 16:16:50.931393 | controller-1 | TASK [vexxhost.containers.cni_plugins : Ensure IPv6 is enabled] ****************
2026-03-05 16:16:50.931397 | controller-1 | Thursday 05 March 2026  16:16:50 +0000 (0:00:01.001)       0:01:26.382 ********
2026-03-05 16:16:51.663447 | controller-1 | changed: [controller-2]
2026-03-05 16:16:51.663485 | controller-1 | changed: [controller-1]
2026-03-05 16:16:51.663494 | controller-1 | changed: [controller-3]
2026-03-05 16:16:51.663503 | controller-1 |
2026-03-05 16:16:51.663512 | controller-1 | TASK [vexxhost.containers.cni_plugins : Enable kernel modules on-boot] *********
2026-03-05 16:16:51.663529 | controller-1 | Thursday 05 March 2026  16:16:51 +0000 (0:00:00.734)       0:01:27.117 ********
2026-03-05 16:16:52.367418 | controller-1 | changed: [controller-1]
2026-03-05 16:16:52.367468 | controller-1 | changed: [controller-3]
2026-03-05 16:16:52.367557 | controller-1 | changed: [controller-2]
2026-03-05 16:16:52.367636 | controller-1 |
2026-03-05 16:16:52.367743 | controller-1 | TASK [vexxhost.containers.cni_plugins : Enable kernel modules in runtime] ******
2026-03-05 16:16:52.367810 | controller-1 | Thursday 05 March 2026  16:16:52 +0000 (0:00:00.704)       0:01:27.822 ********
2026-03-05 16:16:53.883700 | controller-1 | changed: [controller-2] => (item=br_netfilter)
2026-03-05 16:16:53.883776 | controller-1 | changed: [controller-1] => (item=br_netfilter)
2026-03-05 16:16:53.883824 | controller-1 | changed: [controller-3] => (item=br_netfilter)
2026-03-05 16:16:53.883914 | controller-1 | ok: [controller-1] => (item=ip_tables)
2026-03-05 16:16:53.884001 | controller-1 | ok: [controller-2] => (item=ip_tables)
2026-03-05 16:16:53.884109 | controller-1 | ok: [controller-3] => (item=ip_tables)
2026-03-05 16:16:53.884187 | controller-1 | changed: [controller-1] => (item=ip6_tables)
2026-03-05 16:16:53.884288 | controller-1 | changed: [controller-3] => (item=ip6_tables)
2026-03-05 16:16:53.884357 | controller-1 | changed: [controller-2] => (item=ip6_tables)
2026-03-05 16:16:53.884441 | controller-1 | changed: [controller-1] => (item=nf_conntrack)
2026-03-05 16:16:53.884526 | controller-1 | changed: [controller-3] => (item=nf_conntrack)
2026-03-05 16:16:53.884610 | controller-1 | changed: [controller-2] => (item=nf_conntrack)
2026-03-05 16:16:53.884690 | controller-1 |
2026-03-05 16:16:53.884777 | controller-1 | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-03-05 16:16:53.884863 | controller-1 | Thursday 05 March 2026  16:16:53 +0000 (0:00:01.514)       0:01:29.336 ********
2026-03-05 16:16:54.227353 | controller-1 | ok: [controller-1]
2026-03-05 16:16:54.227431 | controller-1 | ok: [controller-2]
2026-03-05 16:16:54.227444 | controller-1 | ok: [controller-3]
2026-03-05 16:16:54.227542 | controller-1 |
2026-03-05 16:16:54.227627 | controller-1 | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-03-05 16:16:54.227754 | controller-1 | Thursday 05 March 2026  16:16:54 +0000 (0:00:00.345)       0:01:29.682 ********
2026-03-05 16:16:54.323705 | controller-1 | ok: [controller-1] => {
2026-03-05 16:16:54.323780 | controller-1 |     "msg": "https://dl.k8s.io/release/v1.33.4/bin/linux/amd64/kubelet"
2026-03-05 16:16:54.323868 | controller-1 | }
2026-03-05 16:16:54.323956 | controller-1 | ok: [controller-2] => {
2026-03-05 16:16:54.324055 | controller-1 |     "msg": "https://dl.k8s.io/release/v1.33.4/bin/linux/amd64/kubelet"
2026-03-05 16:16:54.324116 | controller-1 | }
2026-03-05 16:16:54.324210 | controller-1 | ok: [controller-3] => {
2026-03-05 16:16:54.324289 | controller-1 |     "msg": "https://dl.k8s.io/release/v1.33.4/bin/linux/amd64/kubelet"
2026-03-05 16:16:54.324369 | controller-1 | }
2026-03-05 16:16:54.324447 | controller-1 |
2026-03-05 16:16:54.324536 | controller-1 | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-03-05 16:16:54.324650 | controller-1 | Thursday 05 March 2026  16:16:54 +0000 (0:00:00.091)       0:01:29.774 ********
2026-03-05 16:17:20.802646 | controller-1 | changed: [controller-3]
2026-03-05 16:17:20.802692 | controller-1 | changed: [controller-2]
2026-03-05 16:17:20.802730 | controller-1 | changed: [controller-1]
2026-03-05 16:17:20.803013 | controller-1 |
2026-03-05 16:17:20.803038 | controller-1 | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-03-05 16:17:20.803042 | controller-1 | Thursday 05 March 2026  16:17:20 +0000 (0:00:26.483)       0:01:56.257 ********
2026-03-05 16:17:20.843663 | controller-1 | skipping: [controller-1]
2026-03-05 16:17:20.888269 | controller-1 | skipping: [controller-2]
2026-03-05 16:17:20.888339 | controller-1 | skipping: [controller-3]
2026-03-05 16:17:20.888428 | controller-1 |
2026-03-05 16:17:20.888518 | controller-1 | TASK [vexxhost.kubernetes.kubelet : Gather variables for each operating system] ***
2026-03-05 16:17:20.889096 | controller-1 | Thursday 05 March 2026  16:17:20 +0000 (0:00:00.085)       0:01:56.343 ********
2026-03-05 16:17:20.971496 | controller-1 | ok: [controller-1] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/kubernetes/roles/kubelet/vars/debian.yml)
2026-03-05 16:17:20.971539 | controller-1 | ok: [controller-2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/kubernetes/roles/kubelet/vars/debian.yml)
2026-03-05 16:17:20.971555 | controller-1 | ok: [controller-3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/kubernetes/roles/kubelet/vars/debian.yml)
2026-03-05 16:17:20.971663 | controller-1 |
2026-03-05 16:17:20.971750 | controller-1 | TASK [vexxhost.kubernetes.kubelet : Install coreutils] *************************
2026-03-05 16:17:20.971840 | controller-1 | Thursday 05 March 2026  16:17:20 +0000 (0:00:00.083)       0:01:56.426 ********
2026-03-05 16:17:21.043919 | controller-1 | skipping: [controller-1]
2026-03-05 16:17:21.044016 | controller-1 | skipping: [controller-2]
2026-03-05 16:17:21.044571 | controller-1 | skipping: [controller-3]
2026-03-05 16:17:21.044600 | controller-1 |
2026-03-05 16:17:21.044607 | controller-1 | TASK [vexxhost.kubernetes.kubelet : Install additional packages] ***************
2026-03-05 16:17:21.044613 | controller-1 | Thursday 05 March 2026  16:17:21 +0000 (0:00:00.072)       0:01:56.498 ********
2026-03-05 16:17:26.527371 | controller-1 | changed: [controller-1]
2026-03-05 16:17:26.527401 | controller-1 | changed: [controller-3]
2026-03-05 16:17:26.527412 | controller-1 | changed: [controller-2]
2026-03-05 16:17:26.527469 | controller-1 |
2026-03-05 16:17:26.527558 | controller-1 | TASK [vexxhost.kubernetes.kubelet : Configure sysctl values] *******************
2026-03-05 16:17:26.527645 | controller-1 | Thursday 05 March 2026  16:17:26 +0000 (0:00:05.481)       0:02:01.979 ********
2026-03-05 16:17:28.997747 | controller-1 | changed: [controller-2] => (item={'name': 'net.ipv4.ip_forward', 'value': 1})
2026-03-05 16:17:28.997797 | controller-1 | changed: [controller-1] => (item={'name': 'net.ipv4.ip_forward', 'value': 1})
2026-03-05 16:17:28.997980 | controller-1 | changed: [controller-3] => (item={'name': 'net.ipv4.ip_forward', 'value': 1})
2026-03-05 16:17:28.998001 | controller-1 | changed: [controller-2] => (item={'name': 'net.bridge.bridge-nf-call-iptables', 'value': 1})
2026-03-05 16:17:28.998073 | controller-1 | changed: [controller-3] => (item={'name': 'net.bridge.bridge-nf-call-iptables', 'value': 1})
2026-03-05 16:17:28.998161 | controller-1 | changed: [controller-1] => (item={'name': 'net.bridge.bridge-nf-call-iptables', 'value': 1})
2026-03-05 16:17:28.998254 | controller-1 | changed: [controller-3] => (item={'name': 'net.bridge.bridge-nf-call-ip6tables', 'value': 1})
2026-03-05 16:17:28.998339 | controller-1 | changed: [controller-2] => (item={'name': 'net.bridge.bridge-nf-call-ip6tables', 'value': 1})
2026-03-05 16:17:28.998431 | controller-1 | changed: [controller-1] => (item={'name': 'net.bridge.bridge-nf-call-ip6tables', 'value': 1})
2026-03-05 16:17:28.998520 | controller-1 | changed: [controller-3] => (item={'name': 'net.ipv4.conf.all.rp_filter', 'value': 0})
2026-03-05 16:17:28.999800 | controller-1 | changed: [controller-2] => (item={'name': 'net.ipv4.conf.all.rp_filter', 'value': 0})
2026-03-05 16:17:28.999831 | controller-1 | changed: [controller-1] => (item={'name': 'net.ipv4.conf.all.rp_filter', 'value': 0})
2026-03-05 16:17:28.999835 | controller-1 | changed: [controller-3] => (item={'name': 'fs.inotify.max_queued_events', 'value': 1048576})
2026-03-05 16:17:28.999838 | controller-1 | changed: [controller-2] => (item={'name': 'fs.inotify.max_queued_events', 'value': 1048576})
2026-03-05 16:17:28.999842 | controller-1 | changed: [controller-1] => (item={'name': 'fs.inotify.max_queued_events', 'value': 1048576})
2026-03-05 16:17:28.999845 | controller-1 | changed: [controller-3] => (item={'name': 'fs.inotify.max_user_instances', 'value': 8192})
2026-03-05 16:17:28.999848 | controller-1 | changed: [controller-2] => (item={'name': 'fs.inotify.max_user_instances', 'value': 8192})
2026-03-05 16:17:28.999851 | controller-1 | changed: [controller-1] => (item={'name': 'fs.inotify.max_user_instances', 'value': 8192})
2026-03-05 16:17:28.999854 | controller-1 | changed: [controller-3] => (item={'name': 'fs.inotify.max_user_watches', 'value': 1048576})
2026-03-05 16:17:28.999857 | controller-1 | changed: [controller-2] => (item={'name': 'fs.inotify.max_user_watches', 'value': 1048576})
2026-03-05 16:17:28.999861 | controller-1 | changed: [controller-1] => (item={'name': 'fs.inotify.max_user_watches', 'value': 1048576})
2026-03-05 16:17:28.999864 | controller-1 |
2026-03-05 16:17:28.999868 | controller-1 | TASK [vexxhost.kubernetes.kubelet : Create folders for kubernetes configuration] ***
2026-03-05 16:17:28.999871 | controller-1 | Thursday 05 March 2026  16:17:28 +0000 (0:00:02.472)       0:02:04.452 ********
2026-03-05 16:17:29.863062 | controller-1 | changed: [controller-1] => (item=/etc/systemd/system/kubelet.service.d)
2026-03-05 16:17:29.868361 | controller-1 | changed: [controller-2] => (item=/etc/systemd/system/kubelet.service.d)
2026-03-05 16:17:29.868374 | controller-1 | changed: [controller-3] => (item=/etc/systemd/system/kubelet.service.d)
2026-03-05 16:17:29.868383 | controller-1 | ok: [controller-1] => (item=/etc/kubernetes)
2026-03-05 16:17:29.868386 | controller-1 | ok: [controller-2] => (item=/etc/kubernetes)
2026-03-05 16:17:29.868389 | controller-1 | ok: [controller-3] => (item=/etc/kubernetes)
2026-03-05 16:17:29.868393 | controller-1 | ok: [controller-1] => (item=/etc/kubernetes/manifests)
2026-03-05 16:17:29.868396 | controller-1 | ok: [controller-3] => (item=/etc/kubernetes/manifests)
2026-03-05 16:17:29.868400 | controller-1 | ok: [controller-2] => (item=/etc/kubernetes/manifests)
2026-03-05 16:17:29.868403 | controller-1 |
2026-03-05 16:17:29.868407 | controller-1 | TASK [vexxhost.kubernetes.kubelet : Add kubelet systemd service config] ********
2026-03-05 16:17:29.868411 | controller-1 | Thursday 05 March 2026  16:17:29 +0000 (0:00:00.865)       0:02:05.318 ********
2026-03-05 16:17:30.588137 | controller-1 | changed: [controller-1]
2026-03-05 16:17:30.588207 | controller-1 | changed: [controller-3]
2026-03-05 16:17:30.588293 | controller-1 | changed: [controller-2]
2026-03-05 16:17:30.588375 | controller-1 |
2026-03-05 16:17:30.588473 | controller-1 | TASK [vexxhost.kubernetes.kubelet : Add kubeadm dropin for kubelet systemd service config] ***
2026-03-05 16:17:30.588566 | controller-1 | Thursday 05 March 2026  16:17:30 +0000 (0:00:00.725)       0:02:06.043 ********
2026-03-05 16:17:31.309648 | controller-1 | changed: [controller-1]
2026-03-05 16:17:31.309703 | controller-1 | changed: [controller-3]
2026-03-05 16:17:31.309844 | controller-1 | changed: [controller-2]
2026-03-05 16:17:31.309959 | controller-1 |
2026-03-05 16:17:31.310091 | controller-1 | TASK [vexxhost.kubernetes.kubelet : Check swap status] *************************
2026-03-05 16:17:31.310216 | controller-1 | Thursday 05 March 2026  16:17:31 +0000 (0:00:00.721)       0:02:06.764 ********
2026-03-05 16:17:31.734532 | controller-1 | ok: [controller-1]
2026-03-05 16:17:31.735456 | controller-1 | ok: [controller-3]
2026-03-05 16:17:31.735506 | controller-1 | ok: [controller-2]
2026-03-05 16:17:31.735518 | controller-1 |
2026-03-05 16:17:31.735530 | controller-1 | TASK [vexxhost.kubernetes.kubelet : Disable swap] ******************************
2026-03-05 16:17:31.735629 | controller-1 | Thursday 05 March 2026  16:17:31 +0000 (0:00:00.424)       0:02:07.189 ********
2026-03-05 16:17:31.781591 | controller-1 | [ERROR]: Task failed: Conditional result (False) was derived from value of type 'str' at '/home/zuul/.ansible/collections/ansible_collections/vexxhost/kubernetes/roles/kubelet/tasks/main.yml:83:11'. Conditionals must have a boolean result.
2026-03-05 16:17:31.781635 | controller-1 |
2026-03-05 16:17:31.781735 | controller-1 | Task failed.
2026-03-05 16:17:31.781820 | controller-1 | Origin: /home/zuul/.ansible/collections/ansible_collections/vexxhost/kubernetes/roles/kubelet/tasks/main.yml:78:7
2026-03-05 16:17:31.781896 | controller-1 |
2026-03-05 16:17:31.781981 | controller-1 | 76       register: kubelet_swapon
2026-03-05 16:17:31.782053 | controller-1 | 77
2026-03-05 16:17:31.782136 | controller-1 | 78     - name: Disable swap
2026-03-05 16:17:31.782217 | controller-1 |          ^ column 7
2026-03-05 16:17:31.782291 | controller-1 |
2026-03-05 16:17:31.782372 | controller-1 | <<< caused by >>>
2026-03-05 16:17:31.782445 | controller-1 |
2026-03-05 16:17:31.782546 | controller-1 | Conditional result (False) was derived from value of type 'str' at '/home/zuul/.ansible/collections/ansible_collections/vexxhost/kubernetes/roles/kubelet/tasks/main.yml:83:11'. Conditionals must have a boolean result.
2026-03-05 16:17:31.782634 | controller-1 | Origin: /home/zuul/.ansible/collections/ansible_collections/vexxhost/kubernetes/roles/kubelet/tasks/main.yml:83:11
2026-03-05 16:17:31.782710 | controller-1 |
2026-03-05 16:17:31.782794 | controller-1 | 81       ignore_errors: "{{ ansible_check_mode }}"
2026-03-05 16:17:31.782873 | controller-1 | 82       when:
2026-03-05 16:17:31.782956 | controller-1 | 83         - kubelet_swapon.stdout
2026-03-05 16:17:31.783058 | controller-1 |              ^ column 11
2026-03-05 16:17:31.783160 | controller-1 |
2026-03-05 16:17:31.783933 | controller-1 | Broken conditionals can be temporarily allowed with the `ALLOW_BROKEN_CONDITIONALS` configuration option.
2026-03-05 16:17:31.783976 | controller-1 |
2026-03-05 16:17:31.783986 | controller-1 | fatal: [controller-1]: FAILED! => {"changed": false, "msg": "Task failed: Conditional result (False) was derived from value of type 'str' at '/home/zuul/.ansible/collections/ansible_collections/vexxhost/kubernetes/roles/kubelet/tasks/main.yml:83:11'. Conditionals must have a boolean result."}
2026-03-05 16:17:31.784737 | controller-1 | fatal: [controller-2]: FAILED! => {"changed": false, "msg": "Task failed: Conditional result (False) was derived from value of type 'str' at '/home/zuul/.ansible/collections/ansible_collections/vexxhost/kubernetes/roles/kubelet/tasks/main.yml:83:11'. Conditionals must have a boolean result."}
2026-03-05 16:17:31.803627 | controller-1 | fatal: [controller-3]: FAILED! => {"changed": false, "msg": "Task failed: Conditional result (False) was derived from value of type 'str' at '/home/zuul/.ansible/collections/ansible_collections/vexxhost/kubernetes/roles/kubelet/tasks/main.yml:83:11'. Conditionals must have a boolean result."}
2026-03-05 16:17:31.815468 | controller-1 | ...ignoring
2026-03-05 16:17:31.815591 | controller-1 | ...ignoring
2026-03-05 16:17:31.815710 | controller-1 | ...ignoring
2026-03-05 16:17:31.816086 | controller-1 |
2026-03-05 16:17:31.816100 | controller-1 | TASK [vexxhost.kubernetes.kubelet : Remove swapfile from /etc/fstab] ***********
2026-03-05 16:17:31.816114 | controller-1 | Thursday 05 March 2026  16:17:31 +0000 (0:00:00.079)       0:02:07.268 ********
2026-03-05 16:17:32.583447 | controller-1 | ok: [controller-1] => (item=swap)
2026-03-05 16:17:32.583491 | controller-1 | ok: [controller-2] => (item=swap)
2026-03-05 16:17:32.583499 | controller-1 | ok: [controller-3] => (item=swap)
2026-03-05 16:17:32.583507 | controller-1 | ok: [controller-1] => (item=none)
2026-03-05 16:17:32.583514 | controller-1 | ok: [controller-2] => (item=none)
2026-03-05 16:17:32.583521 | controller-1 | ok: [controller-3] => (item=none)
2026-03-05 16:17:32.583528 | controller-1 |
2026-03-05 16:17:32.583544 | controller-1 | TASK [vexxhost.kubernetes.kubelet : Create noswap systemd service config file] ***
2026-03-05 16:17:32.583552 | controller-1 | Thursday 05 March 2026  16:17:32 +0000 (0:00:00.766)       0:02:08.034 ********
2026-03-05 16:17:33.303556 | controller-1 | changed: [controller-1]
2026-03-05 16:17:33.303619 | controller-1 | changed: [controller-2]
2026-03-05 16:17:33.303715 | controller-1 | changed: [controller-3]
2026-03-05 16:17:33.303798 | controller-1 |
2026-03-05 16:17:33.303900 | controller-1 | TASK [vexxhost.kubernetes.kubelet : Enable noswap service] *********************
2026-03-05 16:17:33.303986 | controller-1 | Thursday 05 March 2026  16:17:33 +0000 (0:00:00.723)       0:02:08.758 ********
2026-03-05 16:17:34.115695 | controller-1 | changed: [controller-1]
2026-03-05 16:17:34.115746 | controller-1 | changed: [controller-2]
2026-03-05 16:17:34.115802 | controller-1 | changed: [controller-3]
2026-03-05 16:17:34.115883 | controller-1 |
2026-03-05 16:17:34.116783 | controller-1 | TASK [vexxhost.kubernetes.kubelet : Force any restarts if necessary] ***********
2026-03-05 16:17:34.116807 | controller-1 | Thursday 05 March 2026  16:17:34 +0000 (0:00:00.786)       0:02:09.544 ********
2026-03-05 16:17:34.116812 | controller-1 |
2026-03-05 16:17:34.116815 | controller-1 | TASK [vexxhost.kubernetes.kubelet : Force any restarts if necessary] ***********
2026-03-05 16:17:34.116819 | controller-1 | Thursday 05 March 2026  16:17:34 +0000 (0:00:00.012)       0:02:09.557 ********
2026-03-05 16:17:34.116822 | controller-1 |
2026-03-05 16:17:34.116825 | controller-1 | TASK [vexxhost.kubernetes.kubelet : Force any restarts if necessary] ***********
2026-03-05 16:17:34.116829 | controller-1 | Thursday 05 March 2026  16:17:34 +0000 (0:00:00.007)       0:02:09.564 ********
2026-03-05 16:17:34.116832 | controller-1 |
2026-03-05 16:17:34.116835 | controller-1 | RUNNING HANDLER [vexxhost.kubernetes.kubelet : Reload systemd] *****************
2026-03-05 16:17:34.116838 | controller-1 | Thursday 05 March 2026  16:17:34 +0000 (0:00:00.004)       0:02:09.569 ********
2026-03-05 16:17:35.759728 | controller-1 | ok: [controller-2]
2026-03-05 16:17:35.759778 | controller-1 | ok: [controller-1]
2026-03-05 16:17:35.759790 | controller-1 | ok: [controller-3]
2026-03-05 16:17:35.759887 | controller-1 |
2026-03-05 16:17:35.759977 | controller-1 | TASK [vexxhost.kubernetes.kubelet : Enable and start kubelet service] **********
2026-03-05 16:17:35.760065 | controller-1 | Thursday 05 March 2026  16:17:35 +0000 (0:00:01.643)       0:02:11.213 ********
2026-03-05 16:17:36.634161 | controller-1 | changed: [controller-1]
2026-03-05 16:17:36.634214 | controller-1 | changed: [controller-2]
2026-03-05 16:17:36.634226 | controller-1 | changed: [controller-3]
2026-03-05 16:17:36.634326 | controller-1 |
2026-03-05 16:17:36.634418 | controller-1 | TASK [vexxhost.kubernetes.kubelet : Ensure availability of dbus on Debian] *****
2026-03-05 16:17:36.634511 | controller-1 | Thursday 05 March 2026  16:17:36 +0000 (0:00:00.875)       0:02:12.089 ********
2026-03-05 16:17:37.551710 | controller-1 | ok: [controller-3]
2026-03-05 16:17:37.551766 | controller-1 | ok: [controller-1]
2026-03-05 16:17:37.551853 | controller-1 | ok: [controller-2]
2026-03-05 16:17:37.551957 | controller-1 |
2026-03-05 16:17:37.555361 | controller-1 | TASK [vexxhost.kubernetes.kubelet : Configure short hostname] ******************
2026-03-05 16:17:37.555388 | controller-1 | Thursday 05 March 2026  16:17:37 +0000 (0:00:00.914)       0:02:13.003 ********
2026-03-05 16:17:38.439261 | controller-1 | changed: [controller-1]
2026-03-05 16:17:38.439339 | controller-1 | changed: [controller-3]
2026-03-05 16:17:38.439437 | controller-1 | changed: [controller-2]
2026-03-05 16:17:38.439518 | controller-1 |
2026-03-05 16:17:38.439620 | controller-1 | TASK [vexxhost.kubernetes.kubelet : Ensure hostname inside hosts file] *********
2026-03-05 16:17:38.439713 | controller-1 | Thursday 05 March 2026  16:17:38 +0000 (0:00:00.888)       0:02:13.892 ********
2026-03-05 16:17:38.899614 | controller-1 | changed: [controller-2]
2026-03-05 16:17:38.899648 | controller-1 | changed: [controller-1]
2026-03-05 16:17:38.899736 | controller-1 | changed: [controller-3]
2026-03-05 16:17:38.899811 | controller-1 |
2026-03-05 16:17:38.899900 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Remove kubernetes repository] ***********
2026-03-05 16:17:38.899988 | controller-1 | Thursday 05 March 2026  16:17:38 +0000 (0:00:00.462)       0:02:14.354 ********
2026-03-05 16:17:39.459865 | controller-1 | ok: [controller-1]
2026-03-05 16:17:39.459971 | controller-1 | ok: [controller-3]
2026-03-05 16:17:39.460083 | controller-1 | ok: [controller-2]
2026-03-05 16:17:39.460190 | controller-1 |
2026-03-05 16:17:39.460311 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Setup control plane] ********************
2026-03-05 16:17:39.460435 | controller-1 | Thursday 05 March 2026  16:17:39 +0000 (0:00:00.560)       0:02:14.914 ********
2026-03-05 16:17:39.579265 | controller-1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/kubernetes/roles/kubernetes/tasks/control-plane.yml for controller-1, controller-2, controller-3
2026-03-05 16:17:39.579351 | controller-1 |
2026-03-05 16:17:39.579404 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Bootstrap cluster] **********************
2026-03-05 16:17:39.579504 | controller-1 | Thursday 05 March 2026  16:17:39 +0000 (0:00:00.119)       0:02:15.034 ********
2026-03-05 16:17:39.691676 | controller-1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/kubernetes/roles/kubernetes/tasks/bootstrap-cluster.yml for controller-1, controller-2, controller-3
2026-03-05 16:17:39.691707 | controller-1 |
2026-03-05 16:17:39.691828 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Check if any control plane is bootstrapped] ***
2026-03-05 16:17:39.691915 | controller-1 | Thursday 05 March 2026  16:17:39 +0000 (0:00:00.110)       0:02:15.145 ********
2026-03-05 16:17:40.582386 | controller-1 | ok: [controller-1] => (item=controller-1)
2026-03-05 16:17:40.582519 | controller-1 | ok: [controller-2 -> controller-1(162.253.55.206)] => (item=controller-1)
2026-03-05 16:17:40.582760 | controller-1 | ok: [controller-3 -> controller-1(162.253.55.206)] => (item=controller-1)
2026-03-05 16:17:40.582899 | controller-1 | ok: [controller-1 -> controller-2(199.204.45.156)] => (item=controller-2)
2026-03-05 16:17:40.583083 | controller-1 | ok: [controller-2] => (item=controller-2)
2026-03-05 16:17:40.583275 | controller-1 | ok: [controller-3 -> controller-2(199.204.45.156)] => (item=controller-2)
2026-03-05 16:17:40.583519 | controller-1 | ok: [controller-2 -> controller-3(199.204.45.244)] => (item=controller-3)
2026-03-05 16:17:40.583718 | controller-1 | ok: [controller-1 -> controller-3(199.204.45.244)] => (item=controller-3)
2026-03-05 16:17:40.583956 | controller-1 | ok: [controller-3] => (item=controller-3)
2026-03-05 16:17:40.584107 | controller-1 |
2026-03-05 16:17:40.584313 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Pick node from pre-existing cluster] ****
2026-03-05 16:17:40.584594 | controller-1 | Thursday 05 March 2026  16:17:40 +0000 (0:00:00.891)       0:02:16.036 ********
2026-03-05 16:17:40.650323 | controller-1 | skipping: [controller-1]
2026-03-05 16:17:40.650355 | controller-1 | skipping: [controller-2]
2026-03-05 16:17:40.650498 | controller-1 | skipping: [controller-3]
2026-03-05 16:17:40.650612 | controller-1 |
2026-03-05 16:17:40.650846 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Select first node to initialize cluster] ***
2026-03-05 16:17:40.650989 | controller-1 | Thursday 05 March 2026  16:17:40 +0000 (0:00:00.068)       0:02:16.105 ********
2026-03-05 16:17:40.730040 | controller-1 | ok: [controller-1]
2026-03-05 16:17:40.730186 | controller-1 | ok: [controller-2]
2026-03-05 16:17:40.730417 | controller-1 | ok: [controller-3]
2026-03-05 16:17:40.730538 | controller-1 |
2026-03-05 16:17:40.730781 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Print selected bootstrap node] **********
2026-03-05 16:17:40.730910 | controller-1 | Thursday 05 March 2026  16:17:40 +0000 (0:00:00.079)       0:02:16.184 ********
2026-03-05 16:17:40.797890 | controller-1 | ok: [controller-1] => {
2026-03-05 16:17:40.798027 | controller-1 |     "msg": "controller-1"
2026-03-05 16:17:40.798253 | controller-1 | }
2026-03-05 16:17:40.798393 | controller-1 | ok: [controller-2] => {
2026-03-05 16:17:40.798620 | controller-1 |     "msg": "controller-1"
2026-03-05 16:17:40.798737 | controller-1 | }
2026-03-05 16:17:40.798932 | controller-1 | ok: [controller-3] => {
2026-03-05 16:17:40.799093 | controller-1 |     "msg": "controller-1"
2026-03-05 16:17:40.799263 | controller-1 | }
2026-03-05 16:17:40.799485 | controller-1 |
2026-03-05 16:17:40.799672 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Upload cluster configuration for bootstrap node] ***
2026-03-05 16:17:40.799857 | controller-1 | Thursday 05 March 2026  16:17:40 +0000 (0:00:00.067)       0:02:16.252 ********
2026-03-05 16:17:41.508499 | controller-1 | skipping: [controller-2]
2026-03-05 16:17:41.508556 | controller-1 | skipping: [controller-3]
2026-03-05 16:17:41.508682 | controller-1 | changed: [controller-1]
2026-03-05 16:17:41.508792 | controller-1 |
2026-03-05 16:17:41.508937 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Create pki folder] **********************
2026-03-05 16:17:41.509045 | controller-1 | Thursday 05 March 2026  16:17:41 +0000 (0:00:00.710)       0:02:16.963 ********
2026-03-05 16:17:41.587448 | controller-1 | skipping: [controller-1]
2026-03-05 16:17:41.587495 | controller-1 | skipping: [controller-2]
2026-03-05 16:17:41.587505 | controller-1 | skipping: [controller-3]
2026-03-05 16:17:41.587514 | controller-1 |
2026-03-05 16:17:41.587723 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Create kubernetes ca key] ***************
2026-03-05 16:17:41.587762 | controller-1 | Thursday 05 March 2026  16:17:41 +0000 (0:00:00.078)       0:02:17.041 ********
2026-03-05 16:17:41.662870 | controller-1 | skipping: [controller-1]
2026-03-05 16:17:41.662924 | controller-1 | skipping: [controller-2]
2026-03-05 16:17:41.662966 | controller-1 | skipping: [controller-3]
2026-03-05 16:17:41.663056 | controller-1 |
2026-03-05 16:17:41.663142 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Create kubernetes ca cert] **************
2026-03-05 16:17:41.663241 | controller-1 | Thursday 05 March 2026  16:17:41 +0000 (0:00:00.075)       0:02:17.117 ********
2026-03-05 16:17:41.711838 | controller-1 | skipping: [controller-1]
2026-03-05 16:17:41.751974 | controller-1 | skipping: [controller-2]
2026-03-05 16:17:41.752034 | controller-1 | skipping: [controller-3]
2026-03-05 16:17:41.752160 | controller-1 |
2026-03-05 16:17:41.752295 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Create etcd-ca key] *********************
2026-03-05 16:17:41.752436 | controller-1 | Thursday 05 March 2026  16:17:41 +0000 (0:00:00.084)       0:02:17.202 ********
2026-03-05 16:17:41.826731 | controller-1 | skipping: [controller-1]
2026-03-05 16:17:41.826787 | controller-1 | skipping: [controller-2]
2026-03-05 16:17:41.826799 | controller-1 | skipping: [controller-3]
2026-03-05 16:17:41.826895 | controller-1 |
2026-03-05 16:17:41.826996 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Create etcd-ca cert] ********************
2026-03-05 16:17:41.827098 | controller-1 | Thursday 05 March 2026  16:17:41 +0000 (0:00:00.079)       0:02:17.281 ********
2026-03-05 16:17:41.902611 | controller-1 | skipping: [controller-1]
2026-03-05 16:17:41.902712 | controller-1 | skipping: [controller-2]
2026-03-05 16:17:41.903362 | controller-1 | skipping: [controller-3]
2026-03-05 16:17:41.903391 | controller-1 |
2026-03-05 16:17:41.903396 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Create front-proxy-ca key] **************
2026-03-05 16:17:41.903400 | controller-1 | Thursday 05 March 2026  16:17:41 +0000 (0:00:00.075)       0:02:17.357 ********
2026-03-05 16:17:41.979793 | controller-1 | skipping: [controller-1]
2026-03-05 16:17:41.979909 | controller-1 | skipping: [controller-2]
2026-03-05 16:17:41.980034 | controller-1 | skipping: [controller-3]
2026-03-05 16:17:41.980147 | controller-1 |
2026-03-05 16:17:41.980273 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Create front-proxy-ca cert] *************
2026-03-05 16:17:41.980412 | controller-1 | Thursday 05 March 2026  16:17:41 +0000 (0:00:00.075)       0:02:17.432 ********
2026-03-05 16:17:42.050796 | controller-1 | skipping: [controller-1]
2026-03-05 16:17:42.050847 | controller-1 | skipping: [controller-2]
2026-03-05 16:17:42.050991 | controller-1 | skipping: [controller-3]
2026-03-05 16:17:42.051101 | controller-1 |
2026-03-05 16:17:42.051229 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Initialize cluster] *********************
2026-03-05 16:17:42.051400 | controller-1 | Thursday 05 March 2026  16:17:42 +0000 (0:00:00.072)       0:02:17.505 ********
2026-03-05 16:18:21.709433 | controller-1 | changed: [controller-1]
2026-03-05 16:18:21.747547 | controller-1 | skipping: [controller-2]
2026-03-05 16:18:21.747743 | controller-1 | skipping: [controller-3]
2026-03-05 16:18:21.747938 | controller-1 |
2026-03-05 16:18:21.748175 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Check if the node is already part of the cluster] ***
2026-03-05 16:18:21.748374 | controller-1 | Thursday 05 March 2026  16:18:21 +0000 (0:00:39.696)       0:02:57.202 ********
2026-03-05 16:18:22.094344 | controller-1 | ok: [controller-1]
2026-03-05 16:18:22.094500 | controller-1 | ok: [controller-3]
2026-03-05 16:18:22.094680 | controller-1 | ok: [controller-2]
2026-03-05 16:18:22.094851 | controller-1 |
2026-03-05 16:18:22.095046 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Join cluster] ***************************
2026-03-05 16:18:22.095240 | controller-1 | Thursday 05 March 2026  16:18:22 +0000 (0:00:00.345)       0:02:57.548 ********
2026-03-05 16:18:22.188278 | controller-1 | skipping: [controller-1]
2026-03-05 16:18:22.188502 | controller-1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/kubernetes/roles/kubernetes/tasks/join-cluster.yml for controller-2, controller-3
2026-03-05 16:18:22.188689 | controller-1 |
2026-03-05 16:18:22.188908 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Generate control-plane certificates for joining cluster] ***
2026-03-05 16:18:22.189179 | controller-1 | Thursday 05 March 2026  16:18:22 +0000 (0:00:00.094)       0:02:57.643 ********
2026-03-05 16:18:22.648863 | controller-1 | ok: [controller-2 -> controller-1(162.253.55.206)]
2026-03-05 16:18:22.648987 | controller-1 |
2026-03-05 16:18:22.649247 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Retrieve SHA256 certificate hash] *******
2026-03-05 16:18:22.649387 | controller-1 | Thursday 05 March 2026  16:18:22 +0000 (0:00:00.460)       0:02:58.103 ********
2026-03-05 16:18:23.182289 | controller-1 | ok: [controller-2 -> controller-1(162.253.55.206)]
2026-03-05 16:18:23.182339 | controller-1 |
2026-03-05 16:18:23.182352 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Generate token for joining cluster] *****
2026-03-05 16:18:23.182362 | controller-1 | Thursday 05 March 2026  16:18:23 +0000 (0:00:00.532)       0:02:58.635 ********
2026-03-05 16:18:23.810993 | controller-1 | changed: [controller-2 -> controller-1(162.253.55.206)]
2026-03-05 16:18:23.811051 | controller-1 |
2026-03-05 16:18:23.811103 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Upload kubeadm configuration] ***********
2026-03-05 16:18:23.811205 | controller-1 | Thursday 05 March 2026  16:18:23 +0000 (0:00:00.626)       0:02:59.262 ********
2026-03-05 16:18:24.576757 | controller-1 | changed: [controller-3]
2026-03-05 16:18:24.576783 | controller-1 | changed: [controller-2]
2026-03-05 16:18:24.576789 | controller-1 |
2026-03-05 16:18:24.576794 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Join cluster] ***************************
2026-03-05 16:18:24.576799 | controller-1 | Thursday 05 March 2026  16:18:24 +0000 (0:00:00.768)       0:03:00.030 ********
2026-03-05 16:19:31.610370 | controller-1 | changed: [controller-2]
2026-03-05 16:19:31.610427 | controller-1 | changed: [controller-3]
2026-03-05 16:19:31.610542 | controller-1 |
2026-03-05 16:19:31.610675 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Create folder for admin configuration] ***
2026-03-05 16:19:31.610801 | controller-1 | Thursday 05 March 2026  16:19:31 +0000 (0:01:07.033)       0:04:07.064 ********
2026-03-05 16:19:32.101873 | controller-1 | changed: [controller-2]
2026-03-05 16:19:32.101931 | controller-1 | changed: [controller-3]
2026-03-05 16:19:32.102063 | controller-1 | changed: [controller-1]
2026-03-05 16:19:32.102490 | controller-1 |
2026-03-05 16:19:32.102518 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Creating a symlink for admin configuration file] ***
2026-03-05 16:19:32.102524 | controller-1 | Thursday 05 March 2026  16:19:32 +0000 (0:00:00.491)       0:04:07.556 ********
2026-03-05 16:19:32.481762 | controller-1 | changed: [controller-1]
2026-03-05 16:19:32.484011 | controller-1 | changed: [controller-2]
2026-03-05 16:19:32.484307 | controller-1 | changed: [controller-3]
2026-03-05 16:19:32.484609 | controller-1 |
2026-03-05 16:19:32.484914 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Add bash autocomplete for kubectl] ******
2026-03-05 16:19:32.485200 | controller-1 | Thursday 05 March 2026  16:19:32 +0000 (0:00:00.379)       0:04:07.936 ********
2026-03-05 16:19:32.819670 | controller-1 | changed: [controller-1]
2026-03-05 16:19:32.819719 | controller-1 | changed: [controller-2]
2026-03-05 16:19:32.819796 | controller-1 | changed: [controller-3]
2026-03-05 16:19:32.819877 | controller-1 |
2026-03-05 16:19:32.819969 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Install PIP] ****************************
2026-03-05 16:19:32.820077 | controller-1 | Thursday 05 March 2026  16:19:32 +0000 (0:00:00.336)       0:04:08.273 ********
2026-03-05 16:19:32.906577 | controller-1 | skipping: [controller-1]
2026-03-05 16:19:32.906748 | controller-1 | skipping: [controller-2]
2026-03-05 16:19:32.906932 | controller-1 | skipping: [controller-3]
2026-03-05 16:19:32.907100 | controller-1 |
2026-03-05 16:19:32.907336 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Install Kubernetes Python package using pip on supported systems] ***
2026-03-05 16:19:32.907555 | controller-1 | Thursday 05 March 2026  16:19:32 +0000 (0:00:00.086)       0:04:08.359 ********
2026-03-05 16:19:32.979057 | controller-1 | skipping: [controller-1]
2026-03-05 16:19:32.979243 | controller-1 | skipping: [controller-2]
2026-03-05 16:19:32.979466 | controller-1 | skipping: [controller-3]
2026-03-05 16:19:32.979636 | controller-1 |
2026-03-05 16:19:32.979830 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Enable EPEL repository] *****************
2026-03-05 16:19:32.980250 | controller-1 | Thursday 05 March 2026  16:19:32 +0000 (0:00:00.073)       0:04:08.433 ********
2026-03-05 16:19:33.045250 | controller-1 | skipping: [controller-1]
2026-03-05 16:19:33.045306 | controller-1 | skipping: [controller-2]
2026-03-05 16:19:33.045425 | controller-1 | skipping: [controller-3]
2026-03-05 16:19:33.045530 | controller-1 |
2026-03-05 16:19:33.045660 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Install Kubernetes Python package using package manager on supported systems] ***
2026-03-05 16:19:33.045780 | controller-1 | Thursday 05 March 2026  16:19:33 +0000 (0:00:00.066)       0:04:08.500 ********
2026-03-05 16:19:43.190702 | controller-1 | changed: [controller-1]
2026-03-05 16:19:43.190884 | controller-1 | changed: [controller-2]
2026-03-05 16:19:43.191079 | controller-1 | changed: [controller-3]
2026-03-05 16:19:43.191249 | controller-1 |
2026-03-05 16:19:43.191491 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Allow workload on control plane node] ***
2026-03-05 16:19:43.191688 | controller-1 | Thursday 05 March 2026  16:19:43 +0000 (0:00:10.145)       0:04:18.645 ********
2026-03-05 16:19:44.143909 | controller-1 | changed: [controller-3]
2026-03-05 16:19:44.143957 | controller-1 | changed: [controller-1]
2026-03-05 16:19:44.146485 | controller-1 | changed: [controller-2]
2026-03-05 16:19:44.146504 | controller-1 |
2026-03-05 16:19:44.146513 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Remove kube-proxy resources] ************
2026-03-05 16:19:44.146523 | controller-1 | Thursday 05 March 2026  16:19:44 +0000 (0:00:00.951)       0:04:19.596 ********
2026-03-05 16:19:44.192425 | controller-1 | skipping: [controller-1] => (item=DaemonSet)
2026-03-05 16:19:44.192649 | controller-1 | skipping: [controller-1] => (item=ConfigMap)
2026-03-05 16:19:44.193064 | controller-1 | skipping: [controller-1]
2026-03-05 16:19:44.193075 | controller-1 |
2026-03-05 16:19:44.193320 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Start an upgrade] ***********************
2026-03-05 16:19:44.193529 | controller-1 | Thursday 05 March 2026  16:19:44 +0000 (0:00:00.050)       0:04:19.646 ********
2026-03-05 16:19:44.229996 | controller-1 | skipping: [controller-1]
2026-03-05 16:19:44.230153 | controller-1 |
2026-03-05 16:19:44.230360 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Check if the Kubernetes API services is up to date] ***
2026-03-05 16:19:44.230548 | controller-1 | Thursday 05 March 2026  16:19:44 +0000 (0:00:00.037)       0:04:19.684 ********
2026-03-05 16:19:44.302370 | controller-1 | skipping: [controller-1]
2026-03-05 16:19:44.302539 | controller-1 | skipping: [controller-2]
2026-03-05 16:19:44.302719 | controller-1 | skipping: [controller-3]
2026-03-05 16:19:44.302884 | controller-1 |
2026-03-05 16:19:44.303369 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Trigger an upgrade of the Kubernetes API services] ***
2026-03-05 16:19:44.303736 | controller-1 | Thursday 05 March 2026  16:19:44 +0000 (0:00:00.072)       0:04:19.757 ********
2026-03-05 16:19:44.355459 | controller-1 | skipping: [controller-1]
2026-03-05 16:19:44.394410 | controller-1 | skipping: [controller-2]
2026-03-05 16:19:44.394590 | controller-1 | skipping: [controller-3]
2026-03-05 16:19:44.394768 | controller-1 |
2026-03-05 16:19:44.394998 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Setup nodes] ****************************
2026-03-05 16:19:44.395187 | controller-1 | Thursday 05 March 2026  16:19:44 +0000 (0:00:00.091)       0:04:19.849 ********
2026-03-05 16:19:44.460996 | controller-1 | skipping: [controller-1]
2026-03-05 16:19:44.461183 | controller-1 | skipping: [controller-2]
2026-03-05 16:19:44.461433 | controller-1 | skipping: [controller-3]
2026-03-05 16:19:44.461662 | controller-1 |
2026-03-05 16:19:44.461947 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Upgrade & restart Kubelet node for upgrade (if needed)] ***
2026-03-05 16:19:44.462181 | controller-1 | Thursday 05 March 2026  16:19:44 +0000 (0:00:00.066)       0:04:19.915 ********
2026-03-05 16:19:45.418378 | controller-1 | ok: [controller-1]
2026-03-05 16:19:45.811706 | controller-1 | ok: [controller-2]
2026-03-05 16:19:45.811846 | controller-1 | ok: [controller-3]
2026-03-05 16:19:45.812012 | controller-1 |
2026-03-05 16:19:45.812258 | controller-1 | TASK [vexxhost.kubernetes.kubernetes : Set node selector for CoreDNS components] ***
2026-03-05 16:19:45.812451 | controller-1 | Thursday 05 March 2026  16:19:45 +0000 (0:00:01.350)       0:04:21.266 ********
2026-03-05 16:19:46.848459 | controller-1 | changed: [controller-1]
2026-03-05 16:19:46.848488 | controller-1 |
2026-03-05 16:19:46.848494 | controller-1 | PLAY [Install control-plane components] ****************************************
2026-03-05 16:19:46.848500 | controller-1 |
2026-03-05 16:19:46.848505 | controller-1 | TASK [Gathering Facts] *********************************************************
2026-03-05 16:19:46.848510 | controller-1 | Thursday 05 March 2026  16:19:46 +0000 (0:00:01.036)       0:04:22.303 ********
2026-03-05 16:19:48.714165 | controller-1 | ok: [controller-1]
2026-03-05 16:19:48.714222 | controller-1 | ok: [controller-2]
2026-03-05 16:19:48.714289 | controller-1 | ok: [controller-3]
2026-03-05 16:19:48.714550 | controller-1 |
2026-03-05 16:19:48.714643 | controller-1 | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-03-05 16:19:48.714719 | controller-1 | Thursday 05 March 2026  16:19:48 +0000 (0:00:01.864)       0:04:24.168 ********
2026-03-05 16:19:49.087377 | controller-1 | ok: [controller-1]
2026-03-05 16:19:49.087837 | controller-1 | ok: [controller-3]
2026-03-05 16:19:49.087892 | controller-1 | ok: [controller-2]
2026-03-05 16:19:49.088203 | controller-1 |
2026-03-05 16:19:49.088255 | controller-1 | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-03-05 16:19:49.088538 | controller-1 | Thursday 05 March 2026  16:19:49 +0000 (0:00:00.373)       0:04:24.542 ********
2026-03-05 16:19:51.275396 | controller-1 | ok: [controller-3]
2026-03-05 16:19:51.275516 | controller-1 | ok: [controller-1]
2026-03-05 16:19:51.275626 | controller-1 | ok: [controller-2]
2026-03-05 16:19:51.276373 | controller-1 |
2026-03-05 16:19:51.276401 | controller-1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] ***
2026-03-05 16:19:51.276413 | controller-1 | Thursday 05 March 2026  16:19:51 +0000 (0:00:02.188)       0:04:26.730 ********
2026-03-05 16:19:51.637523 | controller-1 | ok: [controller-2]
2026-03-05 16:19:51.637683 | controller-1 | ok: [controller-1]
2026-03-05 16:19:51.637971 | controller-1 | ok: [controller-3]
2026-03-05 16:19:51.638162 | controller-1 |
2026-03-05 16:19:51.638371 | controller-1 | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-03-05 16:19:51.638570 | controller-1 | Thursday 05 March 2026  16:19:51 +0000 (0:00:00.361)       0:04:27.092 ********
2026-03-05 16:19:51.706925 | controller-1 | ok: [controller-1] => {
2026-03-05 16:19:51.707093 | controller-1 |     "msg": "https://get.helm.sh/helm-v3.11.2-linux-amd64.tar.gz"
2026-03-05 16:19:51.707275 | controller-1 | }
2026-03-05 16:19:51.707527 | controller-1 | ok: [controller-2] => {
2026-03-05 16:19:51.707722 | controller-1 |     "msg": "https://get.helm.sh/helm-v3.11.2-linux-amd64.tar.gz"
2026-03-05 16:19:51.707923 | controller-1 | }
2026-03-05 16:19:51.708118 | controller-1 | ok: [controller-3] => {
2026-03-05 16:19:51.708306 | controller-1 |     "msg": "https://get.helm.sh/helm-v3.11.2-linux-amd64.tar.gz"
2026-03-05 16:19:51.708482 | controller-1 | }
2026-03-05 16:19:51.708644 | controller-1 |
2026-03-05 16:19:51.708844 | controller-1 | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-03-05 16:19:51.709032 | controller-1 | Thursday 05 March 2026  16:19:51 +0000 (0:00:00.069)       0:04:27.161 ********
2026-03-05 16:19:52.942974 | controller-1 | changed: [controller-1]
2026-03-05 16:19:52.943065 | controller-1 | changed: [controller-2]
2026-03-05 16:19:52.943206 | controller-1 | changed: [controller-3]
2026-03-05 16:19:52.943358 | controller-1 |
2026-03-05 16:19:52.943501 | controller-1 | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-03-05 16:19:52.943640 | controller-1 | Thursday 05 March 2026  16:19:52 +0000 (0:00:01.236)       0:04:28.397 ********
2026-03-05 16:19:55.749419 | controller-1 | changed: [controller-3]
2026-03-05 16:19:55.749599 | controller-1 | changed: [controller-2]
2026-03-05 16:19:55.749784 | controller-1 | changed: [controller-1]
2026-03-05 16:19:55.749956 | controller-1 |
2026-03-05 16:19:55.750155 | controller-1 | TASK [vexxhost.kubernetes.helm : Remove Helm repository] ***********************
2026-03-05 16:19:55.750350 | controller-1 | Thursday 05 March 2026  16:19:55 +0000 (0:00:02.806)       0:04:31.204 ********
2026-03-05 16:19:56.178365 | controller-1 | ok: [controller-1]
2026-03-05 16:19:56.178460 | controller-1 | ok: [controller-3]
2026-03-05 16:19:56.178571 | controller-1 | ok: [controller-2]
2026-03-05 16:19:56.178673 | controller-1 |
2026-03-05 16:19:56.178785 | controller-1 | TASK [vexxhost.kubernetes.helm : Add bash autocomplete for helm] ***************
2026-03-05 16:19:56.178893 | controller-1 | Thursday 05 March 2026  16:19:56 +0000 (0:00:00.428)       0:04:31.632 ********
2026-03-05 16:19:56.517716 | controller-1 | changed: [controller-1]
2026-03-05 16:19:56.517890 | controller-1 | changed: [controller-2]
2026-03-05 16:19:56.518082 | controller-1 | changed: [controller-3]
2026-03-05 16:19:56.518326 | controller-1 |
2026-03-05 16:19:56.518546 | controller-1 | TASK [vexxhost.kubernetes.helm : Get Helm plugins dir] *************************
2026-03-05 16:19:56.518747 | controller-1 | Thursday 05 March 2026  16:19:56 +0000 (0:00:00.338)       0:04:31.971 ********
2026-03-05 16:19:56.920859 | controller-1 | ok: [controller-1]
2026-03-05 16:19:56.920916 | controller-1 | ok: [controller-2]
2026-03-05 16:19:56.920926 | controller-1 | ok: [controller-3]
2026-03-05 16:19:56.920935 | controller-1 |
2026-03-05 16:19:56.920944 | controller-1 | TASK [vexxhost.kubernetes.helm : Create Helm plugins directory if it does not exist] ***
2026-03-05 16:19:56.920953 | controller-1 | Thursday 05 March 2026  16:19:56 +0000 (0:00:00.402)       0:04:32.373 ********
2026-03-05 16:19:57.242792 | controller-1 | changed: [controller-2]
2026-03-05 16:19:57.243115 | controller-1 | changed: [controller-1]
2026-03-05 16:19:57.243436 | controller-1 | changed: [controller-3]
2026-03-05 16:19:57.243678 | controller-1 |
2026-03-05 16:19:57.244014 | controller-1 | TASK [vexxhost.kubernetes.helm : Remove Helm diff plugin installed by kubernetes.core.helm_plugin] ***
2026-03-05 16:19:57.244277 | controller-1 | Thursday 05 March 2026  16:19:57 +0000 (0:00:00.323)       0:04:32.697 ********
2026-03-05 16:19:57.560697 | controller-1 | ok: [controller-1]
2026-03-05 16:19:57.561008 | controller-1 | ok: [controller-2]
2026-03-05 16:19:57.561256 | controller-1 | ok: [controller-3]
2026-03-05 16:19:57.561487 | controller-1 |
2026-03-05 16:19:57.561740 | controller-1 | TASK [Install plugin] **********************************************************
2026-03-05 16:19:57.561989 | controller-1 | Thursday 05 March 2026  16:19:57 +0000 (0:00:00.317)       0:04:33.015 ********
2026-03-05 16:19:57.647818 | controller-1 | included: vexxhost.containers.download_artifact for controller-1, controller-2, controller-3
2026-03-05 16:19:57.648074 | controller-1 |
2026-03-05 16:19:57.648334 | controller-1 | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-03-05 16:19:57.648585 | controller-1 | Thursday 05 March 2026  16:19:57 +0000 (0:00:00.087)       0:04:33.102 ********
2026-03-05 16:19:57.712480 | controller-1 | ok: [controller-1] => {
2026-03-05 16:19:57.712817 | controller-1 |     "msg": "https://github.com/databus23/helm-diff/releases/download/v3.8.1/helm-diff-linux-amd64.tgz"
2026-03-05 16:19:57.713079 | controller-1 | }
2026-03-05 16:19:57.713437 | controller-1 | ok: [controller-2] => {
2026-03-05 16:19:57.713787 | controller-1 |     "msg": "https://github.com/databus23/helm-diff/releases/download/v3.8.1/helm-diff-linux-amd64.tgz"
2026-03-05 16:19:57.714034 | controller-1 | }
2026-03-05 16:19:57.714281 | controller-1 | ok: [controller-3] => {
2026-03-05 16:19:57.714547 | controller-1 |     "msg": "https://github.com/databus23/helm-diff/releases/download/v3.8.1/helm-diff-linux-amd64.tgz"
2026-03-05 16:19:57.714910 | controller-1 | }
2026-03-05 16:19:57.715150 | controller-1 |
2026-03-05 16:19:57.715647 | controller-1 | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-03-05 16:19:57.716047 | controller-1 | Thursday 05 March 2026  16:19:57 +0000 (0:00:00.064)       0:04:33.167 ********
2026-03-05 16:19:59.433498 | controller-1 | changed: [controller-2]
2026-03-05 16:19:59.433553 | controller-1 | changed: [controller-3]
2026-03-05 16:19:59.433563 | controller-1 | changed: [controller-1]
2026-03-05 16:19:59.433571 | controller-1 |
2026-03-05 16:19:59.433579 | controller-1 | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-03-05 16:19:59.433588 | controller-1 | Thursday 05 March 2026  16:19:59 +0000 (0:00:01.719)       0:04:34.886 ********
2026-03-05 16:20:01.601372 | controller-1 | changed: [controller-2]
2026-03-05 16:20:01.601667 | controller-1 | changed: [controller-3]
2026-03-05 16:20:01.601920 | controller-1 | changed: [controller-1]
2026-03-05 16:20:01.602243 | controller-1 |
2026-03-05 16:20:01.602536 | controller-1 | TASK [vexxhost.kubernetes.upload_helm_chart : Include help chart upload method tasks] ***
2026-03-05 16:20:01.602806 | controller-1 | Thursday 05 March 2026  16:20:01 +0000 (0:00:02.169)       0:04:37.055 ********
2026-03-05 16:20:01.718498 | controller-1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/kubernetes/roles/upload_helm_chart/tasks/synchronize.yml for controller-1, controller-2, controller-3
2026-03-05 16:20:01.718853 | controller-1 |
2026-03-05 16:20:01.719163 | controller-1 | TASK [vexxhost.kubernetes.upload_helm_chart : Upload Helm chart] ***************
2026-03-05 16:20:01.719512 | controller-1 | Thursday 05 March 2026  16:20:01 +0000 (0:00:00.117)       0:04:37.172 ********
2026-03-05 16:20:02.552828 | controller-1 | changed: [controller-1]
2026-03-05 16:20:02.553114 | controller-1 |
2026-03-05 16:20:02.553818 | controller-1 | TASK [vexxhost.kubernetes.cilium : Get current Kubernetes version] *************
2026-03-05 16:20:02.554294 | controller-1 | Thursday 05 March 2026  16:20:02 +0000 (0:00:00.834)       0:04:38.007 ********
2026-03-05 16:20:03.526126 | controller-1 | ok: [controller-1]
2026-03-05 16:20:03.526164 | controller-1 |
2026-03-05 16:20:03.526173 | controller-1 | TASK [vexxhost.kubernetes.cilium : Deploy Helm chart] **************************
2026-03-05 16:20:03.526182 | controller-1 | Thursday 05 March 2026  16:20:03 +0000 (0:00:00.972)       0:04:38.979 ********
2026-03-05 16:20:05.657238 | controller-1 | changed: [controller-1]
2026-03-05 16:20:05.657433 | controller-1 |
2026-03-05 16:20:05.657605 | controller-1 | PLAY [Wait for all nodes to become Ready] **************************************
2026-03-05 16:20:05.657754 | controller-1 |
2026-03-05 16:20:05.657914 | controller-1 | TASK [Gathering Facts] *********************************************************
2026-03-05 16:20:05.658072 | controller-1 | Thursday 05 March 2026  16:20:05 +0000 (0:00:02.132)       0:04:41.112 ********
2026-03-05 16:20:06.510685 | controller-1 | ok: [controller-1]
2026-03-05 16:20:06.510999 | controller-1 | ok: [controller-3]
2026-03-05 16:20:06.511266 | controller-1 | ok: [controller-2]
2026-03-05 16:20:06.511591 | controller-1 |
2026-03-05 16:20:06.511870 | controller-1 | TASK [Wait for node to be Ready] ***********************************************
2026-03-05 16:20:06.512135 | controller-1 | Thursday 05 March 2026  16:20:06 +0000 (0:00:00.852)       0:04:41.965 ********
2026-03-05 16:20:26.161122 | controller-1 | [ERROR]: Task failed: Module failed: 500
2026-03-05 16:20:26.161328 | controller-1 | Reason: Internal Server Error
2026-03-05 16:20:26.161523 | controller-1 | HTTP response headers: HTTPHeaderDict({'Audit-Id': 'c099cb4b-3135-4899-a9c5-df632775999a', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Kubernetes-Pf-Flowschema-Uid': '8b2e43ac-f197-4892-a526-edbdac2c5aea', 'X-Kubernetes-Pf-Prioritylevel-Uid': '5057696c-3a9b-436b-8fb8-3fe7b9b7b5c8', 'Date': 'Thu, 05 Mar 2026 16:20:26 GMT', 'Content-Length': '122'})
2026-03-05 16:20:26.161699 | controller-1 | HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"etcdserver: request timed out","code":500}\n'
2026-03-05 16:20:26.161856 | controller-1 | Original traceback:
2026-03-05 16:20:26.162035 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/dynamic/client.py", line 55, in inner
2026-03-05 16:20:26.162206 | controller-1 |     resp = func(self, *args, **kwargs)
2026-03-05 16:20:26.162945 | controller-1 |
2026-03-05 16:20:26.164265 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/dynamic/client.py", line 273, in request
2026-03-05 16:20:26.164315 | controller-1 |     api_response = self.client.call_api(
2026-03-05 16:20:26.164475 | controller-1 |         path,
2026-03-05 16:20:26.164790 | controller-1 |     ...<11 lines>...
2026-03-05 16:20:26.165340 | controller-1 |         _request_timeout=params.get('_request_timeout')
2026-03-05 16:20:26.165656 | controller-1 |     )
2026-03-05 16:20:26.168953 | controller-1 |
2026-03-05 16:20:26.168967 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/api_client.py", line 348, in call_api
2026-03-05 16:20:26.168972 | controller-1 |     return self.__call_api(resource_path, method,
2026-03-05 16:20:26.168977 | controller-1 |            ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.168981 | controller-1 |                            path_params, query_params, header_params,
2026-03-05 16:20:26.168986 | controller-1 |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.169157 | controller-1 |     ...<2 lines>...
2026-03-05 16:20:26.169168 | controller-1 |                            _return_http_data_only, collection_formats,
2026-03-05 16:20:26.169222 | controller-1 |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.170163 | controller-1 |                            _preload_content, _request_timeout, _host)
2026-03-05 16:20:26.170332 | controller-1 |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.170662 | controller-1 |
2026-03-05 16:20:26.170859 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/api_client.py", line 180, in __call_api
2026-03-05 16:20:26.171100 | controller-1 |     response_data = self.request(
2026-03-05 16:20:26.171244 | controller-1 |         method, url, query_params=query_params, headers=header_params,
2026-03-05 16:20:26.171397 | controller-1 |         post_params=post_params, body=body,
2026-03-05 16:20:26.171539 | controller-1 |         _preload_content=_preload_content,
2026-03-05 16:20:26.171682 | controller-1 |         _request_timeout=_request_timeout)
2026-03-05 16:20:26.171824 | controller-1 |
2026-03-05 16:20:26.171980 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/api_client.py", line 373, in request
2026-03-05 16:20:26.172121 | controller-1 |     return self.rest_client.GET(url,
2026-03-05 16:20:26.172538 | controller-1 |            ~~~~~~~~~~~~~~~~~~~~^^^^^
2026-03-05 16:20:26.172700 | controller-1 |                                 query_params=query_params,
2026-03-05 16:20:26.172850 | controller-1 |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.173573 | controller-1 |                                 _preload_content=_preload_content,
2026-03-05 16:20:26.173742 | controller-1 |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.173879 | controller-1 |                                 _request_timeout=_request_timeout,
2026-03-05 16:20:26.174013 | controller-1 |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.174283 | controller-1 |                                 headers=headers)
2026-03-05 16:20:26.174653 | controller-1 |                                 ^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.174924 | controller-1 |
2026-03-05 16:20:26.175061 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/rest.py", line 244, in GET
2026-03-05 16:20:26.175336 | controller-1 |     return self.request("GET", url,
2026-03-05 16:20:26.175640 | controller-1 |            ~~~~~~~~~~~~^^^^^^^^^^^^
2026-03-05 16:20:26.176149 | controller-1 |                         headers=headers,
2026-03-05 16:20:26.176520 | controller-1 |                         ^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.176654 | controller-1 |                         _preload_content=_preload_content,
2026-03-05 16:20:26.177057 | controller-1 |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.177188 | controller-1 |                         _request_timeout=_request_timeout,
2026-03-05 16:20:26.177429 | controller-1 |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.177576 | controller-1 |                         query_params=query_params)
2026-03-05 16:20:26.177686 | controller-1 |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.178102 | controller-1 |
2026-03-05 16:20:26.178404 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/rest.py", line 238, in request
2026-03-05 16:20:26.178934 | controller-1 |     raise ApiException(http_resp=r)
2026-03-05 16:20:26.179116 | controller-1 |
2026-03-05 16:20:26.179269 | controller-1 | Task failed: Module failed.
2026-03-05 16:20:26.179532 | controller-1 | Origin: /home/zuul/src/github.com/vexxhost/ansible-collection-kubernetes/extensions/molecule/cluster-upgrade/prepare.yml:25:7
2026-03-05 16:20:26.179635 | controller-1 |
2026-03-05 16:20:26.179878 | controller-1 | 23   become: true
2026-03-05 16:20:26.179985 | controller-1 | 24   tasks:
2026-03-05 16:20:26.180096 | controller-1 | 25     - name: Wait for node to be Ready
2026-03-05 16:20:26.180202 | controller-1 |          ^ column 7
2026-03-05 16:20:26.180304 | controller-1 |
2026-03-05 16:20:26.180413 | controller-1 | <<< caused by >>>
2026-03-05 16:20:26.180513 | controller-1 |
2026-03-05 16:20:26.180618 | controller-1 | 500
2026-03-05 16:20:26.180726 | controller-1 | Reason: Internal Server Error
2026-03-05 16:20:26.181257 | controller-1 | HTTP response headers: HTTPHeaderDict({'Audit-Id': 'c099cb4b-3135-4899-a9c5-df632775999a', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Kubernetes-Pf-Flowschema-Uid': '8b2e43ac-f197-4892-a526-edbdac2c5aea', 'X-Kubernetes-Pf-Prioritylevel-Uid': '5057696c-3a9b-436b-8fb8-3fe7b9b7b5c8', 'Date': 'Thu, 05 Mar 2026 16:20:26 GMT', 'Content-Length': '122'})
2026-03-05 16:20:26.183564 | controller-1 | HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"etcdserver: request timed out","code":500}\n'
2026-03-05 16:20:26.183714 | controller-1 | Original traceback:
2026-03-05 16:20:26.183832 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/dynamic/client.py", line 55, in inner
2026-03-05 16:20:26.183940 | controller-1 |     resp = func(self, *args, **kwargs)
2026-03-05 16:20:26.184039 | controller-1 |
2026-03-05 16:20:26.184153 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/dynamic/client.py", line 273, in request
2026-03-05 16:20:26.184260 | controller-1 |     api_response = self.client.call_api(
2026-03-05 16:20:26.184363 | controller-1 |         path,
2026-03-05 16:20:26.184466 | controller-1 |     ...<11 lines>...
2026-03-05 16:20:26.184574 | controller-1 |         _request_timeout=params.get('_request_timeout')
2026-03-05 16:20:26.184679 | controller-1 |     )
2026-03-05 16:20:26.184775 | controller-1 |
2026-03-05 16:20:26.184887 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/api_client.py", line 348, in call_api
2026-03-05 16:20:26.184995 | controller-1 |     return self.__call_api(resource_path, method,
2026-03-05 16:20:26.185097 | controller-1 |            ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.185223 | controller-1 |                            path_params, query_params, header_params,
2026-03-05 16:20:26.185328 | controller-1 |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.185489 | controller-1 |     ...<2 lines>...
2026-03-05 16:20:26.185600 | controller-1 |                            _return_http_data_only, collection_formats,
2026-03-05 16:20:26.185701 | controller-1 |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.185807 | controller-1 |                            _preload_content, _request_timeout, _host)
2026-03-05 16:20:26.185952 | controller-1 |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.186053 | controller-1 |
2026-03-05 16:20:26.186169 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/api_client.py", line 180, in __call_api
2026-03-05 16:20:26.186275 | controller-1 |     response_data = self.request(
2026-03-05 16:20:26.186410 | controller-1 |         method, url, query_params=query_params, headers=header_params,
2026-03-05 16:20:26.186534 | controller-1 |         post_params=post_params, body=body,
2026-03-05 16:20:26.187070 | controller-1 |         _preload_content=_preload_content,
2026-03-05 16:20:26.187080 | controller-1 |         _request_timeout=_request_timeout)
2026-03-05 16:20:26.187085 | controller-1 |
2026-03-05 16:20:26.187089 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/api_client.py", line 373, in request
2026-03-05 16:20:26.187214 | controller-1 |     return self.rest_client.GET(url,
2026-03-05 16:20:26.187354 | controller-1 |            ~~~~~~~~~~~~~~~~~~~~^^^^^
2026-03-05 16:20:26.187461 | controller-1 |                                 query_params=query_params,
2026-03-05 16:20:26.188014 | controller-1 |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.188124 | controller-1 |                                 _preload_content=_preload_content,
2026-03-05 16:20:26.188228 | controller-1 |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.188506 | controller-1 |                                 _request_timeout=_request_timeout,
2026-03-05 16:20:26.188694 | controller-1 |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.188857 | controller-1 |                                 headers=headers)
2026-03-05 16:20:26.189024 | controller-1 |                                 ^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.189221 | controller-1 |
2026-03-05 16:20:26.189386 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/rest.py", line 244, in GET
2026-03-05 16:20:26.189540 | controller-1 |     return self.request("GET", url,
2026-03-05 16:20:26.189644 | controller-1 |            ~~~~~~~~~~~~^^^^^^^^^^^^
2026-03-05 16:20:26.189748 | controller-1 |                         headers=headers,
2026-03-05 16:20:26.189850 | controller-1 |                         ^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.189956 | controller-1 |                         _preload_content=_preload_content,
2026-03-05 16:20:26.190072 | controller-1 |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.190178 | controller-1 |                         _request_timeout=_request_timeout,
2026-03-05 16:20:26.192098 | controller-1 |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.192238 | controller-1 |                         query_params=query_params)
2026-03-05 16:20:26.192352 | controller-1 |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.192451 | controller-1 |
2026-03-05 16:20:26.192566 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/rest.py", line 238, in request
2026-03-05 16:20:26.192674 | controller-1 |     raise ApiException(http_resp=r)
2026-03-05 16:20:26.192773 | controller-1 |
2026-03-05 16:20:26.193069 | controller-1 | fatal: [controller-2 -> controller-1(162.253.55.206)]: FAILED! => {"changed": false, "msg": "Task failed: Module failed: 500\nReason: Internal Server Error\nHTTP response headers: HTTPHeaderDict({'Audit-Id': 'c099cb4b-3135-4899-a9c5-df632775999a', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Kubernetes-Pf-Flowschema-Uid': '8b2e43ac-f197-4892-a526-edbdac2c5aea', 'X-Kubernetes-Pf-Prioritylevel-Uid': '5057696c-3a9b-436b-8fb8-3fe7b9b7b5c8', 'Date': 'Thu, 05 Mar 2026 16:20:26 GMT', 'Content-Length': '122'})\nHTTP response body: b'{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"etcdserver: request timed out\",\"code\":500}\\n'\nOriginal traceback: \n  File \"/usr/lib/python3/dist-packages/kubernetes/dynamic/client.py\", line 55, in inner\n    resp = func(self, *args, **kwargs)\n\n  File \"/usr/lib/python3/dist-packages/kubernetes/dynamic/client.py\", line 273, in request\n    api_response = self.client.call_api(\n        path,\n    ...<11 lines>...\n        _request_timeout=params.get('_request_timeout')\n    )\n\n  File \"/usr/lib/python3/dist-packages/kubernetes/client/api_client.py\", line 348, in call_api\n    return self.__call_api(resource_path, method,\n           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^\n                           path_params, query_params, header_params,\n                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n    ...<2 lines>...\n                           _return_http_data_only, collection_formats,\n                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                           _preload_content, _request_timeout, _host)\n                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n  File \"/usr/lib/python3/dist-packages/kubernetes/client/api_client.py\", line 180, in __call_api\n    response_data = self.request(\n        method, url, query_params=query_params, headers=header_params,\n        post_params=post_params, body=body,\n        _preload_content=_preload_content,\n        _request_timeout=_request_timeout)\n\n  File \"/usr/lib/python3/dist-packages/kubernetes/client/api_client.py\", line 373, in request\n    return self.rest_client.GET(url,\n           ~~~~~~~~~~~~~~~~~~~~^^^^^\n                                query_params=query_params,\n                                ^^^^^^^^^^^^^^^^^^^^^^^^^^\n                                _preload_content=_preload_content,\n                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                                _request_timeout=_request_timeout,\n                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                                headers=headers)\n                                ^^^^^^^^^^^^^^^^\n\n  File \"/usr/lib/python3/dist-packages/kubernetes/client/rest.py\", line 244, in GET\n    return self.request(\"GET\", url,\n           ~~~~~~~~~~~~^^^^^^^^^^^^\n                        headers=headers,\n                        ^^^^^^^^^^^^^^^^\n                        _preload_content=_preload_content,\n                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                        _request_timeout=_request_timeout,\n                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                        query_params=query_params)\n                        ^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n  File \"/usr/lib/python3/dist-packages/kubernetes/client/rest.py\", line 238, in request\n    raise ApiException(http_resp=r)"}
2026-03-05 16:20:26.193292 | controller-1 | [ERROR]: Task failed: Module failed: 500
2026-03-05 16:20:26.193402 | controller-1 | Reason: Internal Server Error
2026-03-05 16:20:26.193530 | controller-1 | HTTP response headers: HTTPHeaderDict({'Audit-Id': 'ab4c7d83-a87f-48c0-b8f9-57690f1ee7e0', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Kubernetes-Pf-Flowschema-Uid': '8b2e43ac-f197-4892-a526-edbdac2c5aea', 'X-Kubernetes-Pf-Prioritylevel-Uid': '5057696c-3a9b-436b-8fb8-3fe7b9b7b5c8', 'Date': 'Thu, 05 Mar 2026 16:20:26 GMT', 'Content-Length': '122'})
2026-03-05 16:20:26.193894 | controller-1 | HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"etcdserver: request timed out","code":500}\n'
2026-03-05 16:20:26.194023 | controller-1 | Original traceback:
2026-03-05 16:20:26.194139 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/dynamic/client.py", line 55, in inner
2026-03-05 16:20:26.194246 | controller-1 |     resp = func(self, *args, **kwargs)
2026-03-05 16:20:26.194346 | controller-1 |
2026-03-05 16:20:26.194459 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/dynamic/client.py", line 273, in request
2026-03-05 16:20:26.194608 | controller-1 |     api_response = self.client.call_api(
2026-03-05 16:20:26.194717 | controller-1 |         path,
2026-03-05 16:20:26.194820 | controller-1 |     ...<11 lines>...
2026-03-05 16:20:26.194935 | controller-1 |         _request_timeout=params.get('_request_timeout')
2026-03-05 16:20:26.195039 | controller-1 |     )
2026-03-05 16:20:26.195139 | controller-1 |
2026-03-05 16:20:26.195253 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/api_client.py", line 348, in call_api
2026-03-05 16:20:26.195381 | controller-1 |     return self.__call_api(resource_path, method,
2026-03-05 16:20:26.195485 | controller-1 |            ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.195591 | controller-1 |                            path_params, query_params, header_params,
2026-03-05 16:20:26.195692 | controller-1 |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.195797 | controller-1 |     ...<2 lines>...
2026-03-05 16:20:26.195904 | controller-1 |                            _return_http_data_only, collection_formats,
2026-03-05 16:20:26.196005 | controller-1 |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.196111 | controller-1 |                            _preload_content, _request_timeout, _host)
2026-03-05 16:20:26.196212 | controller-1 |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.196310 | controller-1 |
2026-03-05 16:20:26.196423 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/api_client.py", line 180, in __call_api
2026-03-05 16:20:26.196529 | controller-1 |     response_data = self.request(
2026-03-05 16:20:26.196635 | controller-1 |         method, url, query_params=query_params, headers=header_params,
2026-03-05 16:20:26.196740 | controller-1 |         post_params=post_params, body=body,
2026-03-05 16:20:26.196846 | controller-1 |         _preload_content=_preload_content,
2026-03-05 16:20:26.196951 | controller-1 |         _request_timeout=_request_timeout)
2026-03-05 16:20:26.197049 | controller-1 |
2026-03-05 16:20:26.197165 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/api_client.py", line 373, in request
2026-03-05 16:20:26.197272 | controller-1 |     return self.rest_client.GET(url,
2026-03-05 16:20:26.197374 | controller-1 |            ~~~~~~~~~~~~~~~~~~~~^^^^^
2026-03-05 16:20:26.197479 | controller-1 |                                 query_params=query_params,
2026-03-05 16:20:26.197582 | controller-1 |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.197688 | controller-1 |                                 _preload_content=_preload_content,
2026-03-05 16:20:26.197789 | controller-1 |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.197892 | controller-1 |                                 _request_timeout=_request_timeout,
2026-03-05 16:20:26.197994 | controller-1 |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.198097 | controller-1 |                                 headers=headers)
2026-03-05 16:20:26.198197 | controller-1 |                                 ^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.198350 | controller-1 |
2026-03-05 16:20:26.198463 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/rest.py", line 244, in GET
2026-03-05 16:20:26.198711 | controller-1 |     return self.request("GET", url,
2026-03-05 16:20:26.198869 | controller-1 |            ~~~~~~~~~~~~^^^^^^^^^^^^
2026-03-05 16:20:26.199004 | controller-1 |                         headers=headers,
2026-03-05 16:20:26.199175 | controller-1 |                         ^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.199284 | controller-1 |                         _preload_content=_preload_content,
2026-03-05 16:20:26.199410 | controller-1 |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.199517 | controller-1 |                         _request_timeout=_request_timeout,
2026-03-05 16:20:26.199648 | controller-1 |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.199755 | controller-1 |                         query_params=query_params)
2026-03-05 16:20:26.199856 | controller-1 |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.200078 | controller-1 |
2026-03-05 16:20:26.200255 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/rest.py", line 238, in request
2026-03-05 16:20:26.200457 | controller-1 |     raise ApiException(http_resp=r)
2026-03-05 16:20:26.200560 | controller-1 |
2026-03-05 16:20:26.200666 | controller-1 | Task failed: Module failed.
2026-03-05 16:20:26.200779 | controller-1 | Origin: /home/zuul/src/github.com/vexxhost/ansible-collection-kubernetes/extensions/molecule/cluster-upgrade/prepare.yml:25:7
2026-03-05 16:20:26.200878 | controller-1 |
2026-03-05 16:20:26.200984 | controller-1 | 23   become: true
2026-03-05 16:20:26.201088 | controller-1 | 24   tasks:
2026-03-05 16:20:26.201195 | controller-1 | 25     - name: Wait for node to be Ready
2026-03-05 16:20:26.201300 | controller-1 |          ^ column 7
2026-03-05 16:20:26.201397 | controller-1 |
2026-03-05 16:20:26.201502 | controller-1 | <<< caused by >>>
2026-03-05 16:20:26.201599 | controller-1 |
2026-03-05 16:20:26.201701 | controller-1 | 500
2026-03-05 16:20:26.201807 | controller-1 | Reason: Internal Server Error
2026-03-05 16:20:26.201935 | controller-1 | HTTP response headers: HTTPHeaderDict({'Audit-Id': 'ab4c7d83-a87f-48c0-b8f9-57690f1ee7e0', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Kubernetes-Pf-Flowschema-Uid': '8b2e43ac-f197-4892-a526-edbdac2c5aea', 'X-Kubernetes-Pf-Prioritylevel-Uid': '5057696c-3a9b-436b-8fb8-3fe7b9b7b5c8', 'Date': 'Thu, 05 Mar 2026 16:20:26 GMT', 'Content-Length': '122'})
2026-03-05 16:20:26.202053 | controller-1 | HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"etcdserver: request timed out","code":500}\n'
2026-03-05 16:20:26.202159 | controller-1 | Original traceback:
2026-03-05 16:20:26.202277 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/dynamic/client.py", line 55, in inner
2026-03-05 16:20:26.202385 | controller-1 |     resp = func(self, *args, **kwargs)
2026-03-05 16:20:26.202482 | controller-1 |
2026-03-05 16:20:26.202616 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/dynamic/client.py", line 273, in request
2026-03-05 16:20:26.202722 | controller-1 |     api_response = self.client.call_api(
2026-03-05 16:20:26.202826 | controller-1 |         path,
2026-03-05 16:20:26.202930 | controller-1 |     ...<11 lines>...
2026-03-05 16:20:26.203036 | controller-1 |         _request_timeout=params.get('_request_timeout')
2026-03-05 16:20:26.203139 | controller-1 |     )
2026-03-05 16:20:26.203237 | controller-1 |
2026-03-05 16:20:26.203367 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/api_client.py", line 348, in call_api
2026-03-05 16:20:26.203477 | controller-1 |     return self.__call_api(resource_path, method,
2026-03-05 16:20:26.203583 | controller-1 |            ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.203691 | controller-1 |                            path_params, query_params, header_params,
2026-03-05 16:20:26.203793 | controller-1 |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.203898 | controller-1 |     ...<2 lines>...
2026-03-05 16:20:26.204004 | controller-1 |                            _return_http_data_only, collection_formats,
2026-03-05 16:20:26.204106 | controller-1 |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.204213 | controller-1 |                            _preload_content, _request_timeout, _host)
2026-03-05 16:20:26.204315 | controller-1 |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.204413 | controller-1 |
2026-03-05 16:20:26.204527 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/api_client.py", line 180, in __call_api
2026-03-05 16:20:26.204634 | controller-1 |     response_data = self.request(
2026-03-05 16:20:26.204739 | controller-1 |         method, url, query_params=query_params, headers=header_params,
2026-03-05 16:20:26.204870 | controller-1 |         post_params=post_params, body=body,
2026-03-05 16:20:26.207320 | controller-1 |         _preload_content=_preload_content,
2026-03-05 16:20:26.207329 | controller-1 |         _request_timeout=_request_timeout)
2026-03-05 16:20:26.207337 | controller-1 |
2026-03-05 16:20:26.207341 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/api_client.py", line 373, in request
2026-03-05 16:20:26.207344 | controller-1 |     return self.rest_client.GET(url,
2026-03-05 16:20:26.207347 | controller-1 |            ~~~~~~~~~~~~~~~~~~~~^^^^^
2026-03-05 16:20:26.207351 | controller-1 |                                 query_params=query_params,
2026-03-05 16:20:26.207354 | controller-1 |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.207357 | controller-1 |                                 _preload_content=_preload_content,
2026-03-05 16:20:26.207360 | controller-1 |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.207364 | controller-1 |                                 _request_timeout=_request_timeout,
2026-03-05 16:20:26.207367 | controller-1 |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.207370 | controller-1 |                                 headers=headers)
2026-03-05 16:20:26.207373 | controller-1 |                                 ^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.207376 | controller-1 |
2026-03-05 16:20:26.207380 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/rest.py", line 244, in GET
2026-03-05 16:20:26.207383 | controller-1 |     return self.request("GET", url,
2026-03-05 16:20:26.207386 | controller-1 |            ~~~~~~~~~~~~^^^^^^^^^^^^
2026-03-05 16:20:26.207389 | controller-1 |                         headers=headers,
2026-03-05 16:20:26.207392 | controller-1 |                         ^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.207395 | controller-1 |                         _preload_content=_preload_content,
2026-03-05 16:20:26.207398 | controller-1 |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.207402 | controller-1 |                         _request_timeout=_request_timeout,
2026-03-05 16:20:26.207405 | controller-1 |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.207408 | controller-1 |                         query_params=query_params)
2026-03-05 16:20:26.207411 | controller-1 |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.207414 | controller-1 |
2026-03-05 16:20:26.207417 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/rest.py", line 238, in request
2026-03-05 16:20:26.207474 | controller-1 |     raise ApiException(http_resp=r)
2026-03-05 16:20:26.207574 | controller-1 |
2026-03-05 16:20:26.207871 | controller-1 | fatal: [controller-3 -> controller-1(162.253.55.206)]: FAILED! => {"changed": false, "msg": "Task failed: Module failed: 500\nReason: Internal Server Error\nHTTP response headers: HTTPHeaderDict({'Audit-Id': 'ab4c7d83-a87f-48c0-b8f9-57690f1ee7e0', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Kubernetes-Pf-Flowschema-Uid': '8b2e43ac-f197-4892-a526-edbdac2c5aea', 'X-Kubernetes-Pf-Prioritylevel-Uid': '5057696c-3a9b-436b-8fb8-3fe7b9b7b5c8', 'Date': 'Thu, 05 Mar 2026 16:20:26 GMT', 'Content-Length': '122'})\nHTTP response body: b'{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"etcdserver: request timed out\",\"code\":500}\\n'\nOriginal traceback: \n  File \"/usr/lib/python3/dist-packages/kubernetes/dynamic/client.py\", line 55, in inner\n    resp = func(self, *args, **kwargs)\n\n  File \"/usr/lib/python3/dist-packages/kubernetes/dynamic/client.py\", line 273, in request\n    api_response = self.client.call_api(\n        path,\n    ...<11 lines>...\n        _request_timeout=params.get('_request_timeout')\n    )\n\n  File \"/usr/lib/python3/dist-packages/kubernetes/client/api_client.py\", line 348, in call_api\n    return self.__call_api(resource_path, method,\n           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^\n                           path_params, query_params, header_params,\n                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n    ...<2 lines>...\n                           _return_http_data_only, collection_formats,\n                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                           _preload_content, _request_timeout, _host)\n                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n  File \"/usr/lib/python3/dist-packages/kubernetes/client/api_client.py\", line 180, in __call_api\n    response_data = self.request(\n        method, url, query_params=query_params, headers=header_params,\n        post_params=post_params, body=body,\n        _preload_content=_preload_content,\n        _request_timeout=_request_timeout)\n\n  File \"/usr/lib/python3/dist-packages/kubernetes/client/api_client.py\", line 373, in request\n    return self.rest_client.GET(url,\n           ~~~~~~~~~~~~~~~~~~~~^^^^^\n                                query_params=query_params,\n                                ^^^^^^^^^^^^^^^^^^^^^^^^^^\n                                _preload_content=_preload_content,\n                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                                _request_timeout=_request_timeout,\n                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                                headers=headers)\n                                ^^^^^^^^^^^^^^^^\n\n  File \"/usr/lib/python3/dist-packages/kubernetes/client/rest.py\", line 244, in GET\n    return self.request(\"GET\", url,\n           ~~~~~~~~~~~~^^^^^^^^^^^^\n                        headers=headers,\n                        ^^^^^^^^^^^^^^^^\n                        _preload_content=_preload_content,\n                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                        _request_timeout=_request_timeout,\n                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                        query_params=query_params)\n                        ^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n  File \"/usr/lib/python3/dist-packages/kubernetes/client/rest.py\", line 238, in request\n    raise ApiException(http_resp=r)"}
2026-03-05 16:20:26.207990 | controller-1 | [ERROR]: Task failed: Module failed: 500
2026-03-05 16:20:26.208095 | controller-1 | Reason: Internal Server Error
2026-03-05 16:20:26.208222 | controller-1 | HTTP response headers: HTTPHeaderDict({'Audit-Id': '5d4b7164-aecf-419e-b7c2-72daa5a9892b', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Kubernetes-Pf-Flowschema-Uid': '8b2e43ac-f197-4892-a526-edbdac2c5aea', 'X-Kubernetes-Pf-Prioritylevel-Uid': '5057696c-3a9b-436b-8fb8-3fe7b9b7b5c8', 'Date': 'Thu, 05 Mar 2026 16:20:26 GMT', 'Content-Length': '122'})
2026-03-05 16:20:26.208339 | controller-1 | HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"etcdserver: request timed out","code":500}\n'
2026-03-05 16:20:26.208444 | controller-1 | Original traceback:
2026-03-05 16:20:26.208556 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/dynamic/client.py", line 55, in inner
2026-03-05 16:20:26.208663 | controller-1 |     resp = func(self, *args, **kwargs)
2026-03-05 16:20:26.208799 | controller-1 |
2026-03-05 16:20:26.208915 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/dynamic/client.py", line 273, in request
2026-03-05 16:20:26.209049 | controller-1 |     api_response = self.client.call_api(
2026-03-05 16:20:26.209154 | controller-1 |         path,
2026-03-05 16:20:26.209356 | controller-1 |     ...<11 lines>...
2026-03-05 16:20:26.209467 | controller-1 |         _request_timeout=params.get('_request_timeout')
2026-03-05 16:20:26.209570 | controller-1 |     )
2026-03-05 16:20:26.209668 | controller-1 |
2026-03-05 16:20:26.209781 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/api_client.py", line 348, in call_api
2026-03-05 16:20:26.209888 | controller-1 |     return self.__call_api(resource_path, method,
2026-03-05 16:20:26.209990 | controller-1 |            ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.210096 | controller-1 |                            path_params, query_params, header_params,
2026-03-05 16:20:26.210199 | controller-1 |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.210302 | controller-1 |     ...<2 lines>...
2026-03-05 16:20:26.210410 | controller-1 |                            _return_http_data_only, collection_formats,
2026-03-05 16:20:26.210513 | controller-1 |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.210636 | controller-1 |                            _preload_content, _request_timeout, _host)
2026-03-05 16:20:26.210738 | controller-1 |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.210835 | controller-1 |
2026-03-05 16:20:26.210947 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/api_client.py", line 180, in __call_api
2026-03-05 16:20:26.211060 | controller-1 |     response_data = self.request(
2026-03-05 16:20:26.211167 | controller-1 |         method, url, query_params=query_params, headers=header_params,
2026-03-05 16:20:26.211271 | controller-1 |         post_params=post_params, body=body,
2026-03-05 16:20:26.211396 | controller-1 |         _preload_content=_preload_content,
2026-03-05 16:20:26.211502 | controller-1 |         _request_timeout=_request_timeout)
2026-03-05 16:20:26.211600 | controller-1 |
2026-03-05 16:20:26.211712 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/api_client.py", line 373, in request
2026-03-05 16:20:26.211817 | controller-1 |     return self.rest_client.GET(url,
2026-03-05 16:20:26.211919 | controller-1 |            ~~~~~~~~~~~~~~~~~~~~^^^^^
2026-03-05 16:20:26.212024 | controller-1 |                                 query_params=query_params,
2026-03-05 16:20:26.212125 | controller-1 |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.212231 | controller-1 |                                 _preload_content=_preload_content,
2026-03-05 16:20:26.212333 | controller-1 |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.212438 | controller-1 |                                 _request_timeout=_request_timeout,
2026-03-05 16:20:26.212540 | controller-1 |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.212644 | controller-1 |                                 headers=headers)
2026-03-05 16:20:26.212745 | controller-1 |                                 ^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.212843 | controller-1 |
2026-03-05 16:20:26.212956 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/rest.py", line 244, in GET
2026-03-05 16:20:26.213062 | controller-1 |     return self.request("GET", url,
2026-03-05 16:20:26.213164 | controller-1 |            ~~~~~~~~~~~~^^^^^^^^^^^^
2026-03-05 16:20:26.213267 | controller-1 |                         headers=headers,
2026-03-05 16:20:26.213369 | controller-1 |                         ^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.213474 | controller-1 |                         _preload_content=_preload_content,
2026-03-05 16:20:26.213578 | controller-1 |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.213689 | controller-1 |                         _request_timeout=_request_timeout,
2026-03-05 16:20:26.213792 | controller-1 |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.213896 | controller-1 |                         query_params=query_params)
2026-03-05 16:20:26.213998 | controller-1 |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.214095 | controller-1 |
2026-03-05 16:20:26.214207 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/rest.py", line 238, in request
2026-03-05 16:20:26.214313 | controller-1 |     raise ApiException(http_resp=r)
2026-03-05 16:20:26.214410 | controller-1 |
2026-03-05 16:20:26.214515 | controller-1 | Task failed: Module failed.
2026-03-05 16:20:26.214675 | controller-1 | Origin: /home/zuul/src/github.com/vexxhost/ansible-collection-kubernetes/extensions/molecule/cluster-upgrade/prepare.yml:25:7
2026-03-05 16:20:26.214776 | controller-1 |
2026-03-05 16:20:26.214887 | controller-1 | 23   become: true
2026-03-05 16:20:26.214991 | controller-1 | 24   tasks:
2026-03-05 16:20:26.215098 | controller-1 | 25     - name: Wait for node to be Ready
2026-03-05 16:20:26.215201 | controller-1 |          ^ column 7
2026-03-05 16:20:26.215316 | controller-1 |
2026-03-05 16:20:26.215425 | controller-1 | <<< caused by >>>
2026-03-05 16:20:26.215523 | controller-1 |
2026-03-05 16:20:26.215625 | controller-1 | 500
2026-03-05 16:20:26.215730 | controller-1 | Reason: Internal Server Error
2026-03-05 16:20:26.215857 | controller-1 | HTTP response headers: HTTPHeaderDict({'Audit-Id': '5d4b7164-aecf-419e-b7c2-72daa5a9892b', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Kubernetes-Pf-Flowschema-Uid': '8b2e43ac-f197-4892-a526-edbdac2c5aea', 'X-Kubernetes-Pf-Prioritylevel-Uid': '5057696c-3a9b-436b-8fb8-3fe7b9b7b5c8', 'Date': 'Thu, 05 Mar 2026 16:20:26 GMT', 'Content-Length': '122'})
2026-03-05 16:20:26.215974 | controller-1 | HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"etcdserver: request timed out","code":500}\n'
2026-03-05 16:20:26.216078 | controller-1 | Original traceback:
2026-03-05 16:20:26.216190 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/dynamic/client.py", line 55, in inner
2026-03-05 16:20:26.216296 | controller-1 |     resp = func(self, *args, **kwargs)
2026-03-05 16:20:26.216393 | controller-1 |
2026-03-05 16:20:26.216505 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/dynamic/client.py", line 273, in request
2026-03-05 16:20:26.216616 | controller-1 |     api_response = self.client.call_api(
2026-03-05 16:20:26.216719 | controller-1 |         path,
2026-03-05 16:20:26.216823 | controller-1 |     ...<11 lines>...
2026-03-05 16:20:26.216929 | controller-1 |         _request_timeout=params.get('_request_timeout')
2026-03-05 16:20:26.217031 | controller-1 |     )
2026-03-05 16:20:26.217129 | controller-1 |
2026-03-05 16:20:26.217242 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/api_client.py", line 348, in call_api
2026-03-05 16:20:26.217355 | controller-1 |     return self.__call_api(resource_path, method,
2026-03-05 16:20:26.217458 | controller-1 |            ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.217564 | controller-1 |                            path_params, query_params, header_params,
2026-03-05 16:20:26.217665 | controller-1 |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.217770 | controller-1 |     ...<2 lines>...
2026-03-05 16:20:26.217877 | controller-1 |                            _return_http_data_only, collection_formats,
2026-03-05 16:20:26.217979 | controller-1 |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.218084 | controller-1 |                            _preload_content, _request_timeout, _host)
2026-03-05 16:20:26.218187 | controller-1 |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.218285 | controller-1 |
2026-03-05 16:20:26.218397 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/api_client.py", line 180, in __call_api
2026-03-05 16:20:26.218504 | controller-1 |     response_data = self.request(
2026-03-05 16:20:26.218628 | controller-1 |         method, url, query_params=query_params, headers=header_params,
2026-03-05 16:20:26.218734 | controller-1 |         post_params=post_params, body=body,
2026-03-05 16:20:26.218838 | controller-1 |         _preload_content=_preload_content,
2026-03-05 16:20:26.218941 | controller-1 |         _request_timeout=_request_timeout)
2026-03-05 16:20:26.219038 | controller-1 |
2026-03-05 16:20:26.219150 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/api_client.py", line 373, in request
2026-03-05 16:20:26.219256 | controller-1 |     return self.rest_client.GET(url,
2026-03-05 16:20:26.219377 | controller-1 |            ~~~~~~~~~~~~~~~~~~~~^^^^^
2026-03-05 16:20:26.219483 | controller-1 |                                 query_params=query_params,
2026-03-05 16:20:26.219585 | controller-1 |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.219689 | controller-1 |                                 _preload_content=_preload_content,
2026-03-05 16:20:26.219790 | controller-1 |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.219895 | controller-1 |                                 _request_timeout=_request_timeout,
2026-03-05 16:20:26.219996 | controller-1 |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.220099 | controller-1 |                                 headers=headers)
2026-03-05 16:20:26.220201 | controller-1 |                                 ^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.220298 | controller-1 |
2026-03-05 16:20:26.220411 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/rest.py", line 244, in GET
2026-03-05 16:20:26.220517 | controller-1 |     return self.request("GET", url,
2026-03-05 16:20:26.220620 | controller-1 |            ~~~~~~~~~~~~^^^^^^^^^^^^
2026-03-05 16:20:26.220724 | controller-1 |                         headers=headers,
2026-03-05 16:20:26.220824 | controller-1 |                         ^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.220928 | controller-1 |                         _preload_content=_preload_content,
2026-03-05 16:20:26.221030 | controller-1 |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.221134 | controller-1 |                         _request_timeout=_request_timeout,
2026-03-05 16:20:26.221237 | controller-1 |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.221341 | controller-1 |                         query_params=query_params)
2026-03-05 16:20:26.221443 | controller-1 |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-05 16:20:26.221541 | controller-1 |
2026-03-05 16:20:26.221653 | controller-1 |   File "/usr/lib/python3/dist-packages/kubernetes/client/rest.py", line 238, in request
2026-03-05 16:20:26.221760 | controller-1 |     raise ApiException(http_resp=r)
2026-03-05 16:20:26.221858 | controller-1 |
2026-03-05 16:20:26.222404 | controller-1 | fatal: [controller-1]: FAILED! => {"changed": false, "msg": "Task failed: Module failed: 500\nReason: Internal Server Error\nHTTP response headers: HTTPHeaderDict({'Audit-Id': '5d4b7164-aecf-419e-b7c2-72daa5a9892b', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Kubernetes-Pf-Flowschema-Uid': '8b2e43ac-f197-4892-a526-edbdac2c5aea', 'X-Kubernetes-Pf-Prioritylevel-Uid': '5057696c-3a9b-436b-8fb8-3fe7b9b7b5c8', 'Date': 'Thu, 05 Mar 2026 16:20:26 GMT', 'Content-Length': '122'})\nHTTP response body: b'{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"etcdserver: request timed out\",\"code\":500}\\n'\nOriginal traceback: \n  File \"/usr/lib/python3/dist-packages/kubernetes/dynamic/client.py\", line 55, in inner\n    resp = func(self, *args, **kwargs)\n\n  File \"/usr/lib/python3/dist-packages/kubernetes/dynamic/client.py\", line 273, in request\n    api_response = self.client.call_api(\n        path,\n    ...<11 lines>...\n        _request_timeout=params.get('_request_timeout')\n    )\n\n  File \"/usr/lib/python3/dist-packages/kubernetes/client/api_client.py\", line 348, in call_api\n    return self.__call_api(resource_path, method,\n           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^\n                           path_params, query_params, header_params,\n                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n    ...<2 lines>...\n                           _return_http_data_only, collection_formats,\n                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                           _preload_content, _request_timeout, _host)\n                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n  File \"/usr/lib/python3/dist-packages/kubernetes/client/api_client.py\", line 180, in __call_api\n    response_data = self.request(\n        method, url, query_params=query_params, headers=header_params,\n        post_params=post_params, body=body,\n        _preload_content=_preload_content,\n        _request_timeout=_request_timeout)\n\n  File \"/usr/lib/python3/dist-packages/kubernetes/client/api_client.py\", line 373, in request\n    return self.rest_client.GET(url,\n           ~~~~~~~~~~~~~~~~~~~~^^^^^\n                                query_params=query_params,\n                                ^^^^^^^^^^^^^^^^^^^^^^^^^^\n                                _preload_content=_preload_content,\n                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                                _request_timeout=_request_timeout,\n                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                                headers=headers)\n                                ^^^^^^^^^^^^^^^^\n\n  File \"/usr/lib/python3/dist-packages/kubernetes/client/rest.py\", line 244, in GET\n    return self.request(\"GET\", url,\n           ~~~~~~~~~~~~^^^^^^^^^^^^\n                        headers=headers,\n                        ^^^^^^^^^^^^^^^^\n                        _preload_content=_preload_content,\n                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                        _request_timeout=_request_timeout,\n                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                        query_params=query_params)\n                        ^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n  File \"/usr/lib/python3/dist-packages/kubernetes/client/rest.py\", line 238, in request\n    raise ApiException(http_resp=r)"}
2026-03-05 16:20:26.222431 | controller-1 |
2026-03-05 16:20:26.222518 | controller-1 | PLAY RECAP *********************************************************************
2026-03-05 16:20:26.222658 | controller-1 | controller-1               : ok=115  changed=51   unreachable=0    failed=1    skipped=34   rescued=0    ignored=3
2026-03-05 16:20:26.222775 | controller-1 | controller-2               : ok=113  changed=49   unreachable=0    failed=1    skipped=31   rescued=0    ignored=2
2026-03-05 16:20:26.222889 | controller-1 | controller-3               : ok=110  changed=48   unreachable=0    failed=1    skipped=31   rescued=0    ignored=2
2026-03-05 16:20:26.222988 | controller-1 |
2026-03-05 16:20:26.223089 | controller-1 |
2026-03-05 16:20:26.223194 | controller-1 | TASKS RECAP ********************************************************************
2026-03-05 16:20:26.223325 | controller-1 | Thursday 05 March 2026  16:20:26 +0000 (0:00:19.681)       0:05:01.646 ********
2026-03-05 16:20:26.223430 | controller-1 | ===============================================================================
2026-03-05 16:20:26.223538 | controller-1 | vexxhost.kubernetes.kubernetes : Join cluster -------------------------- 67.03s
2026-03-05 16:20:26.223646 | controller-1 | vexxhost.kubernetes.kubernetes : Initialize cluster -------------------- 39.70s
2026-03-05 16:20:26.223754 | controller-1 | vexxhost.containers.download_artifact : Download item ------------------ 26.48s
2026-03-05 16:20:26.223868 | controller-1 | vexxhost.containers.download_artifact : Download item ------------------ 25.46s
2026-03-05 16:20:26.223976 | controller-1 | Wait for node to be Ready ---------------------------------------------- 19.68s
2026-03-05 16:20:26.224091 | controller-1 | vexxhost.kubernetes.kubernetes : Install Kubernetes Python package using package manager on supported systems -- 10.15s
2026-03-05 16:20:26.224202 | controller-1 | vexxhost.containers.containerd : Install AppArmor packages -------------- 8.71s
2026-03-05 16:20:26.224311 | controller-1 | vexxhost.kubernetes.kubelet : Install additional packages --------------- 5.48s
2026-03-05 16:20:26.224419 | controller-1 | Install Debian packages ------------------------------------------------- 5.31s
2026-03-05 16:20:26.224526 | controller-1 | vexxhost.containers.download_artifact : Extract archive ----------------- 4.02s
2026-03-05 16:20:26.224673 | controller-1 | vexxhost.containers.download_artifact : Extract archive ----------------- 3.94s
2026-03-05 16:20:26.224815 | controller-1 | vexxhost.kubernetes.kubelet : Reload systemd ---------------------------- 3.62s
2026-03-05 16:20:26.224941 | controller-1 | vexxhost.containers.download_artifact : Extract archive ----------------- 2.81s
2026-03-05 16:20:26.225053 | controller-1 | vexxhost.containers.download_artifact : Extract archive ----------------- 2.72s
2026-03-05 16:20:26.225161 | controller-1 | vexxhost.kubernetes.kubelet : Configure sysctl values ------------------- 2.47s
2026-03-05 16:20:26.225268 | controller-1 | Gathering Facts --------------------------------------------------------- 2.24s
2026-03-05 16:20:26.225377 | controller-1 | vexxhost.containers.package : Update state for tar ---------------------- 2.19s
2026-03-05 16:20:26.225484 | controller-1 | vexxhost.containers.download_artifact : Extract archive ----------------- 2.17s
2026-03-05 16:20:26.225594 | controller-1 | vexxhost.kubernetes.cilium : Deploy Helm chart -------------------------- 2.13s
2026-03-05 16:20:26.225700 | controller-1 | Gathering Facts --------------------------------------------------------- 1.87s
2026-03-05 16:20:26.415924 | controller-1 | CRITICAL Ansible return code was 2, command was: ansible-playbook --inventory /home/zuul/.ansible/tmp/molecule.Bs5j.cluster-upgrade/inventory --skip-tags molecule-notest,notest --inventory=/home/zuul/src/github.com/vexxhost/ansible-collection-kubernetes/inventory.yaml /home/zuul/src/github.com/vexxhost/ansible-collection-kubernetes/extensions/molecule/cluster-upgrade/prepare.yml
2026-03-05 16:20:26.416053 | controller-1 | ERROR    [cluster-upgrade > prepare] Executed: Failed
2026-03-05 16:20:26.416149 | controller-1 | ERROR    Ansible return code was 2, command was: ansible-playbook --inventory /home/zuul/.ansible/tmp/molecule.Bs5j.cluster-upgrade/inventory --skip-tags molecule-notest,notest --inventory=/home/zuul/src/github.com/vexxhost/ansible-collection-kubernetes/inventory.yaml /home/zuul/src/github.com/vexxhost/ansible-collection-kubernetes/extensions/molecule/cluster-upgrade/prepare.yml
2026-03-05 16:20:36.748183 | controller-1 | ERROR
2026-03-05 16:20:36.748537 | controller-1 | {
2026-03-05 16:20:36.748606 | controller-1 |   "delta": "0:05:53.304902",
2026-03-05 16:20:36.748654 | controller-1 |   "end": "2026-03-05 16:20:26.535944",
2026-03-05 16:20:36.748698 | controller-1 |   "msg": "non-zero return code",
2026-03-05 16:20:36.748741 | controller-1 |   "rc": 2,
2026-03-05 16:20:36.748787 | controller-1 |   "start": "2026-03-05 16:14:33.231042"
2026-03-05 16:20:36.748830 | controller-1 | }
failure
2026-03-05 16:20:36.766449 | 
2026-03-05 16:20:36.766515 | PLAY RECAP
2026-03-05 16:20:36.766579 | controller-1 | ok: 2 changed: 2 unreachable: 0 failed: 1 skipped: 0 rescued: 0 ignored: 0
2026-03-05 16:20:36.766614 | 
2026-03-05 16:20:36.908611 | RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/run.yaml@main]
2026-03-05 16:20:36.913825 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main]
2026-03-05 16:20:37.520160 | 
2026-03-05 16:20:37.520272 | PLAY [all]
2026-03-05 16:20:37.531201 | 
2026-03-05 16:20:37.531278 | TASK [fetch-output : Set log path for multiple nodes]
2026-03-05 16:20:37.590951 | controller-1 | ok
2026-03-05 16:20:37.597202 | controller-2 | ok
2026-03-05 16:20:37.601380 | controller-3 | ok
2026-03-05 16:20:37.606316 | 
2026-03-05 16:20:37.606415 | TASK [fetch-output : Set log path for single node]
2026-03-05 16:20:37.637896 | controller-1 | skipping: Conditional result was False
2026-03-05 16:20:37.639258 | controller-2 | skipping: Conditional result was False
2026-03-05 16:20:37.649876 | controller-3 | skipping: Conditional result was False
2026-03-05 16:20:37.657059 | 
2026-03-05 16:20:37.657158 | LOOP [fetch-output : Ensure local output dirs]
2026-03-05 16:20:38.052755 | controller-3 -> localhost | changed: "/var/lib/zuul/builds/3f7a9e4c321043949fd99e3d8a4500aa/work/logs/controller-3"
2026-03-05 16:20:38.059248 | controller-1 -> localhost | changed: "/var/lib/zuul/builds/3f7a9e4c321043949fd99e3d8a4500aa/work/logs/controller-1"
2026-03-05 16:20:38.078503 | controller-2 -> localhost | changed: "/var/lib/zuul/builds/3f7a9e4c321043949fd99e3d8a4500aa/work/logs/controller-2"
2026-03-05 16:20:38.249003 | controller-3 -> localhost | changed: "/var/lib/zuul/builds/3f7a9e4c321043949fd99e3d8a4500aa/work/artifacts"
2026-03-05 16:20:38.265078 | controller-1 -> localhost | ok: "/var/lib/zuul/builds/3f7a9e4c321043949fd99e3d8a4500aa/work/artifacts"
2026-03-05 16:20:38.292360 | controller-2 -> localhost | ok: "/var/lib/zuul/builds/3f7a9e4c321043949fd99e3d8a4500aa/work/artifacts"
2026-03-05 16:20:38.456645 | controller-3 -> localhost | changed: "/var/lib/zuul/builds/3f7a9e4c321043949fd99e3d8a4500aa/work/docs"
2026-03-05 16:20:38.465312 | controller-1 -> localhost | ok: "/var/lib/zuul/builds/3f7a9e4c321043949fd99e3d8a4500aa/work/docs"
2026-03-05 16:20:38.465386 | controller-1 -> localhost | changed: All items complete
2026-03-05 16:20:38.465420 | 
2026-03-05 16:20:38.494711 | controller-2 -> localhost | ok: "/var/lib/zuul/builds/3f7a9e4c321043949fd99e3d8a4500aa/work/docs"
2026-03-05 16:20:38.515981 | 
2026-03-05 16:20:38.516843 | LOOP [fetch-output : Collect logs, artifacts and docs]
2026-03-05 16:20:39.189408 | controller-2 | changed: .d..t...... ./
2026-03-05 16:20:39.189699 | controller-2 | changed: All items complete
2026-03-05 16:20:39.189743 | 
2026-03-05 16:20:39.219581 | controller-3 | changed: .d..t...... ./
2026-03-05 16:20:39.248574 | controller-1 | changed: .d..t...... ./
2026-03-05 16:20:39.661388 | controller-2 | changed: .d..t...... ./
2026-03-05 16:20:39.693502 | controller-3 | ok
2026-03-05 16:20:39.755027 | controller-1 | changed: .d..t...... ./
2026-03-05 16:20:40.148690 | controller-2 | changed: .d..t...... ./
2026-03-05 16:20:40.201426 | controller-3 | ok
2026-03-05 16:20:40.280714 | controller-1 | changed: .d..t...... ./
2026-03-05 16:20:40.280865 | controller-1 | changed: All items complete
2026-03-05 16:20:40.280901 | 
2026-03-05 16:20:40.308998 | 
2026-03-05 16:20:40.309163 | LOOP [merge-output-to-logs : Move artifacts and docs to logs dir]
2026-03-05 16:20:40.788326 | controller-1 -> localhost | ok: Item: artifacts Runtime: 0:00:00.007954
2026-03-05 16:20:41.021826 | controller-1 -> localhost | ok: Item: docs Runtime: 0:00:00.008368
2026-03-05 16:20:41.042592 | 
2026-03-05 16:20:41.042734 | PLAY [all]
2026-03-05 16:20:41.050672 | 
2026-03-05 16:20:41.050740 | TASK [remove-build-sshkey : Remove the build SSH key from all nodes]
2026-03-05 16:20:41.547144 | controller-1 | changed
2026-03-05 16:20:41.739843 | controller-2 | changed
2026-03-05 16:20:41.743891 | controller-3 | changed
2026-03-05 16:20:41.754917 | 
2026-03-05 16:20:41.754979 | PLAY RECAP
2026-03-05 16:20:41.755030 | controller-1 | ok: 5 changed: 4 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0
2026-03-05 16:20:41.755060 | controller-2 | ok: 4 changed: 3 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0
2026-03-05 16:20:41.755086 | controller-3 | ok: 4 changed: 3 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0
2026-03-05 16:20:41.755105 | 
2026-03-05 16:20:41.871320 | POST-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main]
2026-03-05 16:20:41.878719 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post-logs.yaml@main]
2026-03-05 16:20:42.418115 | 
2026-03-05 16:20:42.418228 | PLAY [localhost]
2026-03-05 16:20:42.428081 | 
2026-03-05 16:20:42.428151 | TASK [Generate Zuul manifest]
2026-03-05 16:20:42.458970 | localhost | ok
2026-03-05 16:20:42.476665 | 
2026-03-05 16:20:42.476740 | TASK [generate-zuul-manifest : Generate Zuul manifest]
2026-03-05 16:20:42.862822 | localhost | changed
2026-03-05 16:20:42.873892 | 
2026-03-05 16:20:42.873968 | TASK [generate-zuul-manifest : Return Zuul manifest URL to Zuul]
2026-03-05 16:20:42.904471 | localhost | ok
2026-03-05 16:20:42.914295 | 
2026-03-05 16:20:42.914359 | TASK [Upload logs]
2026-03-05 16:20:42.933270 | localhost | ok
2026-03-05 16:20:42.989187 | 
2026-03-05 16:20:42.989287 | TASK [Set zuul-log-path fact]
2026-03-05 16:20:43.010963 | localhost | ok
2026-03-05 16:20:43.022620 | 
2026-03-05 16:20:43.022688 | TASK [set-zuul-log-path-fact : Set log path for a build]
2026-03-05 16:20:43.053578 | localhost | ok
2026-03-05 16:20:43.062015 | 
2026-03-05 16:20:43.062079 | TASK [upload-logs : Create log directories]
2026-03-05 16:20:43.475938 | localhost | changed
2026-03-05 16:20:43.483449 | 
2026-03-05 16:20:43.483522 | TASK [upload-logs : Ensure logs are readable before uploading]
2026-03-05 16:20:43.891930 | localhost -> localhost | ok: Runtime: 0:00:00.005875
2026-03-05 16:20:43.897528 | 
2026-03-05 16:20:43.897701 | TASK [upload-logs : Upload logs to log server]
2026-03-05 16:20:44.362523 | localhost | Output suppressed because no_log was given
2026-03-05 16:20:44.366633 | 
2026-03-05 16:20:44.366703 | LOOP [upload-logs : Compress console log and json output]
2026-03-05 16:20:44.413329 | localhost | skipping: Conditional result was False
2026-03-05 16:20:44.420305 | localhost | skipping: Conditional result was False
2026-03-05 16:20:44.431272 | 
2026-03-05 16:20:44.431370 | LOOP [upload-logs : Upload compressed console log and json output]
2026-03-05 16:20:44.473209 | localhost | skipping: Conditional result was False
2026-03-05 16:20:44.473573 | 
2026-03-05 16:20:44.477075 | localhost | skipping: Conditional result was False
2026-03-05 16:20:44.483625 | 
2026-03-05 16:20:44.483719 | LOOP [upload-logs : Upload console log and json output]