2026-08-18 14:40:34.430347 | Job console starting 2026-08-18 14:40:34.454043 | Updating git repos 2026-08-18 14:40:34.489967 | Cloning repos into workspace 2026-08-18 14:40:34.569746 | Restoring repo states 2026-08-18 14:40:34.576574 | Merging changes 2026-08-18 14:40:36.749331 | Checking out repos 2026-08-18 14:40:36.801711 | Preparing playbooks 2026-08-18 14:40:37.957211 | Running Ansible setup 2026-08-18 14:40:43.508620 | PRE-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main] 2026-08-18 14:40:44.169090 | 2026-08-18 14:40:44.169244 | PLAY [localhost] 2026-08-18 14:40:44.177364 | 2026-08-18 14:40:44.177455 | TASK [Gathering Facts] 2026-08-18 14:40:45.141786 | localhost | ok 2026-08-18 14:40:45.149034 | 2026-08-18 14:40:45.149155 | TASK [Setup log path fact] 2026-08-18 14:40:45.168794 | localhost | ok 2026-08-18 14:40:45.183576 | 2026-08-18 14:40:45.183710 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-08-18 14:40:45.212740 | localhost | ok 2026-08-18 14:40:45.218780 | 2026-08-18 14:40:45.218849 | TASK [emit-job-header : Print job information] 2026-08-18 14:40:45.257857 | # Job Information 2026-08-18 14:40:45.257998 | Ansible Version: 2.16.18 2026-08-18 14:40:45.258045 | Job: ansible-collection-ceph-molecule-ha-ubuntu-noble 2026-08-18 14:40:45.258078 | Pipeline: check 2026-08-18 14:40:45.258109 | Executor: 2d72f0692154 2026-08-18 14:40:45.258139 | Triggered by: https://github.com/vexxhost/ansible-collection-ceph/pull/123 2026-08-18 14:40:45.258172 | Event ID: 5876e6e0-9b12-11f1-83bd-8c5895bbc7af 2026-08-18 14:40:45.262008 | 2026-08-18 14:40:45.262095 | LOOP [emit-job-header : Print node information] 2026-08-18 14:40:45.397997 | localhost | ok: 2026-08-18 14:40:45.398233 | localhost | # Node Information 2026-08-18 14:40:45.398275 | localhost | Inventory Hostname: node1 2026-08-18 14:40:45.398309 | localhost | Hostname: np0000190022 2026-08-18 14:40:45.398339 | localhost | Username: zuul 2026-08-18 14:40:45.398373 | localhost | Distro: Ubuntu 24.04 2026-08-18 14:40:45.398403 | localhost | Provider: yul1 2026-08-18 14:40:45.398432 | localhost | Region: ca-ymq-1 2026-08-18 14:40:45.398477 | localhost | Label: ubuntu-noble 2026-08-18 14:40:45.398506 | localhost | Product Name: OpenStack Nova 2026-08-18 14:40:45.398535 | localhost | Interface IP: 199.204.45.38 2026-08-18 14:40:45.445805 | localhost | ok: 2026-08-18 14:40:45.445910 | localhost | # Node Information 2026-08-18 14:40:45.445967 | localhost | Inventory Hostname: node2 2026-08-18 14:40:45.446014 | localhost | Hostname: np0000190023 2026-08-18 14:40:45.446057 | localhost | Username: zuul 2026-08-18 14:40:45.446099 | localhost | Distro: Ubuntu 24.04 2026-08-18 14:40:45.446141 | localhost | Provider: yul1 2026-08-18 14:40:45.446183 | localhost | Region: ca-ymq-1 2026-08-18 14:40:45.446223 | localhost | Label: ubuntu-noble 2026-08-18 14:40:45.446264 | localhost | Product Name: OpenStack Nova 2026-08-18 14:40:45.446306 | localhost | Interface IP: 199.204.45.44 2026-08-18 14:40:45.496112 | localhost | ok: 2026-08-18 14:40:45.496226 | localhost | # Node Information 2026-08-18 14:40:45.496261 | localhost | Inventory Hostname: node3 2026-08-18 14:40:45.496289 | localhost | Hostname: np0000190024 2026-08-18 14:40:45.496317 | localhost | Username: zuul 2026-08-18 14:40:45.496344 | localhost | Distro: Ubuntu 24.04 2026-08-18 14:40:45.496370 | localhost | Provider: yul1 2026-08-18 14:40:45.496397 | localhost | Region: ca-ymq-1 2026-08-18 14:40:45.496422 | localhost | Label: ubuntu-noble 2026-08-18 14:40:45.496447 | localhost | Product Name: OpenStack Nova 2026-08-18 14:40:45.496472 | localhost | Interface IP: 199.204.45.169 2026-08-18 14:40:45.513375 | 2026-08-18 14:40:45.513619 | TASK [log-inventory : Ensure Zuul Ansible directory exists] 2026-08-18 14:40:45.896292 | localhost -> localhost | changed 2026-08-18 14:40:45.903141 | 2026-08-18 14:40:45.903236 | TASK [log-inventory : Copy ansible inventory to logs dir] 2026-08-18 14:40:46.879692 | localhost -> localhost | changed 2026-08-18 14:40:46.891323 | 2026-08-18 14:40:46.891433 | PLAY [all] 2026-08-18 14:40:46.901686 | 2026-08-18 14:40:46.901813 | TASK [add-build-sshkey : Check to see if ssh key was already created for this build] 2026-08-18 14:40:47.125923 | node1 -> localhost | ok 2026-08-18 14:40:47.140751 | 2026-08-18 14:40:47.140859 | TASK [add-build-sshkey : Create a new key in workspace based on build UUID] 2026-08-18 14:40:47.211725 | node1 | ok 2026-08-18 14:40:47.277532 | node2 | ok 2026-08-18 14:40:47.282402 | node3 | ok 2026-08-18 14:40:47.304898 | node1 | included: /var/lib/zuul/builds/be0cdab9cf3446448d16157e8a3bf679/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml 2026-08-18 14:40:47.304949 | node2 | included: /var/lib/zuul/builds/be0cdab9cf3446448d16157e8a3bf679/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml 2026-08-18 14:40:47.304994 | node3 | included: /var/lib/zuul/builds/be0cdab9cf3446448d16157e8a3bf679/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml 2026-08-18 14:40:47.312208 | 2026-08-18 14:40:47.312272 | TASK [add-build-sshkey : Create Temp SSH key] 2026-08-18 14:40:50.205698 | node1 -> localhost | Generating public/private rsa key pair. 2026-08-18 14:40:50.205912 | node1 -> localhost | Your identification has been saved in /var/lib/zuul/builds/be0cdab9cf3446448d16157e8a3bf679/work/be0cdab9cf3446448d16157e8a3bf679_id_rsa 2026-08-18 14:40:50.205954 | node1 -> localhost | Your public key has been saved in /var/lib/zuul/builds/be0cdab9cf3446448d16157e8a3bf679/work/be0cdab9cf3446448d16157e8a3bf679_id_rsa.pub 2026-08-18 14:40:50.205986 | node1 -> localhost | The key fingerprint is: 2026-08-18 14:40:50.206017 | node1 -> localhost | SHA256:XwN4QzqWFDVAXk08LXZx1a0jhRxixdVyPwMj6kmsl4k zuul-build-sshkey 2026-08-18 14:40:50.206046 | node1 -> localhost | The key's randomart image is: 2026-08-18 14:40:50.206074 | node1 -> localhost | +---[RSA 3072]----+ 2026-08-18 14:40:50.206102 | node1 -> localhost | | .=+BB+=oo*| 2026-08-18 14:40:50.206130 | node1 -> localhost | | o B +Xo=.=| 2026-08-18 14:40:50.206158 | node1 -> localhost | | O =..=o+.| 2026-08-18 14:40:50.206186 | node1 -> localhost | | . * o. oo.| 2026-08-18 14:40:50.206213 | node1 -> localhost | | S + o. .o| 2026-08-18 14:40:50.206240 | node1 -> localhost | | E B . . | 2026-08-18 14:40:50.206267 | node1 -> localhost | | . . | 2026-08-18 14:40:50.206294 | node1 -> localhost | | | 2026-08-18 14:40:50.206322 | node1 -> localhost | | | 2026-08-18 14:40:50.206350 | node1 -> localhost | +----[SHA256]-----+ 2026-08-18 14:40:50.206410 | node1 -> localhost | ok: Runtime: 0:00:02.538703 2026-08-18 14:40:50.213011 | 2026-08-18 14:40:50.213074 | TASK [add-build-sshkey : Remote setup ssh keys (linux)] 2026-08-18 14:40:50.250143 | node1 | ok 2026-08-18 14:40:50.255349 | node2 | ok 2026-08-18 14:40:50.259559 | node3 | ok 2026-08-18 14:40:50.277462 | node1 | included: /var/lib/zuul/builds/be0cdab9cf3446448d16157e8a3bf679/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml 2026-08-18 14:40:50.277520 | node2 | included: /var/lib/zuul/builds/be0cdab9cf3446448d16157e8a3bf679/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml 2026-08-18 14:40:50.277544 | node3 | included: /var/lib/zuul/builds/be0cdab9cf3446448d16157e8a3bf679/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml 2026-08-18 14:40:50.290456 | 2026-08-18 14:40:50.290525 | TASK [add-build-sshkey : Remove previously added zuul-build-sshkey] 2026-08-18 14:40:50.319626 | node1 | skipping: Conditional result was False 2026-08-18 14:40:50.321652 | node2 | skipping: Conditional result was False 2026-08-18 14:40:50.332557 | node3 | skipping: Conditional result was False 2026-08-18 14:40:50.343256 | 2026-08-18 14:40:50.343322 | TASK [add-build-sshkey : Enable access via build key on all nodes] 2026-08-18 14:40:50.903366 | node2 | changed 2026-08-18 14:40:50.908276 | node3 | changed 2026-08-18 14:40:50.912869 | node1 | changed 2026-08-18 14:40:50.918108 | 2026-08-18 14:40:50.918169 | TASK [add-build-sshkey : Make sure user has a .ssh] 2026-08-18 14:40:51.139384 | node1 | ok 2026-08-18 14:40:51.169408 | node2 | ok 2026-08-18 14:40:51.175774 | node3 | ok 2026-08-18 14:40:51.181600 | 2026-08-18 14:40:51.181666 | TASK [add-build-sshkey : Install build private key as SSH key on all nodes] 2026-08-18 14:40:51.725865 | node1 | changed 2026-08-18 14:40:51.756126 | node2 | changed 2026-08-18 14:40:51.761075 | node3 | changed 2026-08-18 14:40:51.767554 | 2026-08-18 14:40:51.767617 | TASK [add-build-sshkey : Install build public key as SSH key on all nodes] 2026-08-18 14:40:52.310327 | node1 | changed 2026-08-18 14:40:52.331503 | node2 | changed 2026-08-18 14:40:52.391435 | node3 | changed 2026-08-18 14:40:52.397811 | 2026-08-18 14:40:52.397879 | TASK [add-build-sshkey : Remote setup ssh keys (windows)] 2026-08-18 14:40:52.426306 | node1 | skipping: Conditional result was False 2026-08-18 14:40:52.437697 | node2 | skipping: Conditional result was False 2026-08-18 14:40:52.438151 | node3 | skipping: Conditional result was False 2026-08-18 14:40:52.447694 | 2026-08-18 14:40:52.447785 | TASK [remove-zuul-sshkey : Remove master key from local agent] 2026-08-18 14:40:52.838759 | node1 -> localhost | changed 2026-08-18 14:40:52.864862 | 2026-08-18 14:40:52.864942 | TASK [add-build-sshkey : Add back temp key] 2026-08-18 14:40:53.184485 | node1 -> localhost | Identity added: /var/lib/zuul/builds/be0cdab9cf3446448d16157e8a3bf679/work/be0cdab9cf3446448d16157e8a3bf679_id_rsa (zuul-build-sshkey) 2026-08-18 14:40:53.184735 | node1 -> localhost | ok: Runtime: 0:00:00.012867 2026-08-18 14:40:53.192294 | 2026-08-18 14:40:53.192378 | TASK [add-build-sshkey : Verify we can still SSH to all nodes] 2026-08-18 14:40:53.539611 | node1 | ok 2026-08-18 14:40:53.544519 | node2 | ok 2026-08-18 14:40:53.564094 | node3 | ok 2026-08-18 14:40:53.571327 | 2026-08-18 14:40:53.571415 | TASK [add-build-sshkey : Verify we can still SSH to all nodes (windows)] 2026-08-18 14:40:53.656889 | node1 | skipping: Conditional result was False 2026-08-18 14:40:53.658169 | node2 | skipping: Conditional result was False 2026-08-18 14:40:53.678824 | node3 | skipping: Conditional result was False 2026-08-18 14:40:53.704223 | 2026-08-18 14:40:53.704298 | TASK [prepare-workspace : Start zuul_console daemon.] 2026-08-18 14:40:54.038947 | node1 | ok 2026-08-18 14:40:54.043861 | node2 | ok 2026-08-18 14:40:54.049870 | node3 | ok 2026-08-18 14:40:54.056533 | 2026-08-18 14:40:54.056594 | TASK [prepare-workspace : Synchronize src repos to workspace directory.] 2026-08-18 14:40:54.787931 | node3 | Output suppressed because no_log was given 2026-08-18 14:40:54.805105 | node1 | Output suppressed because no_log was given 2026-08-18 14:40:54.836974 | node2 | Output suppressed because no_log was given 2026-08-18 14:40:54.855630 | 2026-08-18 14:40:54.855696 | LOOP [ensure-output-dirs : Empty Zuul Output directories by removing them] 2026-08-18 14:40:55.080829 | node1 | ok: "logs" 2026-08-18 14:40:55.081141 | node1 | ok: All items complete 2026-08-18 14:40:55.081170 | 2026-08-18 14:40:55.087035 | node2 | ok: "logs" 2026-08-18 14:40:55.103850 | node3 | ok: "logs" 2026-08-18 14:40:55.272652 | node1 | ok: "artifacts" 2026-08-18 14:40:55.279525 | node2 | ok: "artifacts" 2026-08-18 14:40:55.309061 | node3 | ok: "artifacts" 2026-08-18 14:40:55.465211 | node1 | ok: "docs" 2026-08-18 14:40:55.476493 | node2 | ok: "docs" 2026-08-18 14:40:55.504970 | node3 | ok: "docs" 2026-08-18 14:40:55.505058 | node3 | ok: All items complete 2026-08-18 14:40:55.505083 | 2026-08-18 14:40:55.522310 | 2026-08-18 14:40:55.522460 | LOOP [ensure-output-dirs : Ensure Zuul Output directories exist] 2026-08-18 14:40:55.757683 | node2 | changed: "logs" 2026-08-18 14:40:55.762840 | node1 | changed: "logs" 2026-08-18 14:40:55.767705 | node3 | changed: "logs" 2026-08-18 14:40:55.949848 | node2 | changed: "artifacts" 2026-08-18 14:40:55.972999 | node3 | changed: "artifacts" 2026-08-18 14:40:55.980004 | node1 | changed: "artifacts" 2026-08-18 14:40:56.138459 | node2 | changed: "docs" 2026-08-18 14:40:56.171375 | node1 | changed: "docs" 2026-08-18 14:40:56.171464 | node1 | changed: All items complete 2026-08-18 14:40:56.171500 | 2026-08-18 14:40:56.187819 | node3 | changed: "docs" 2026-08-18 14:40:56.211235 | 2026-08-18 14:40:56.211353 | PLAY RECAP 2026-08-18 14:40:56.211400 | localhost | ok: 6 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-08-18 14:40:56.211430 | node1 | ok: 15 changed: 8 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0 2026-08-18 14:40:56.211455 | node2 | ok: 11 changed: 5 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0 2026-08-18 14:40:56.211478 | node3 | ok: 11 changed: 5 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0 2026-08-18 14:40:56.211497 | 2026-08-18 14:40:56.351054 | PRE-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main] 2026-08-18 14:40:56.369199 | PRE-RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main] 2026-08-18 14:40:57.012616 | 2026-08-18 14:40:57.012753 | PLAY [all] 2026-08-18 14:40:57.025442 | 2026-08-18 14:40:57.025520 | TASK [setup-uv : Extract archive] 2026-08-18 14:40:59.492230 | node2 | changed 2026-08-18 14:40:59.634375 | node1 | changed 2026-08-18 14:40:59.663561 | node3 | changed 2026-08-18 14:40:59.671562 | 2026-08-18 14:40:59.671685 | TASK [setup-uv : Print version] 2026-08-18 14:41:00.073096 | node1 | uv 0.8.13 2026-08-18 14:41:00.076256 | node3 | uv 0.8.13 2026-08-18 14:41:00.077101 | node2 | uv 0.8.13 2026-08-18 14:41:00.296583 | node2 | ok: Runtime: 0:00:00.011729 2026-08-18 14:41:00.306116 | node1 | ok: Runtime: 0:00:00.016972 2026-08-18 14:41:00.314657 | node3 | ok: Runtime: 0:00:00.015901 2026-08-18 14:41:00.333014 | 2026-08-18 14:41:00.333078 | TASK [Install Ansible collection siblings] 2026-08-18 14:41:00.683689 | node1 | ok: 2026-08-18 14:41:00.683851 | node1 | Skipping community.crypto: no checked-out sibling 2026-08-18 14:41:00.683884 | node1 | Skipping community.docker: no checked-out sibling 2026-08-18 14:41:00.683908 | node1 | Skipping community.general: no checked-out sibling 2026-08-18 14:41:00.683929 | node1 | Skipping ansible.utils: no checked-out sibling 2026-08-18 14:41:00.683951 | node1 | Skipping vexxhost.containers: no checked-out sibling 2026-08-18 14:41:00.685784 | 2026-08-18 14:41:00.685831 | PLAY RECAP 2026-08-18 14:41:00.685877 | node1 | ok: 3 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-08-18 14:41:00.685905 | node2 | ok: 2 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-08-18 14:41:00.685929 | node3 | ok: 2 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-08-18 14:41:00.685962 | 2026-08-18 14:41:00.830956 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main] 2026-08-18 14:41:00.846115 | RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/run.yaml@main] 2026-08-18 14:41:01.447495 | 2026-08-18 14:41:01.447616 | PLAY [all] 2026-08-18 14:41:01.460105 | 2026-08-18 14:41:01.460189 | LOOP [Check inventory directories] 2026-08-18 14:41:01.820376 | node1 | ok: "src/github.com/vexxhost/ansible-collection-ceph" 2026-08-18 14:41:01.995512 | node1 | ok: "src/github.com/vexxhost/ansible-collection-ceph/molecule/default/inventory" 2026-08-18 14:41:02.169044 | node1 | ok: "src/github.com/vexxhost/ansible-collection-ceph/extensions/molecule/default/inventory" 2026-08-18 14:41:02.192450 | 2026-08-18 14:41:02.192710 | LOOP [Copy inventory file for Zuul] 2026-08-18 14:41:03.077509 | node1 | changed: 2026-08-18 14:41:03.077713 | node1 | { 2026-08-18 14:41:03.077741 | node1 | "ansible_loop_var": "item", 2026-08-18 14:41:03.077764 | node1 | "changed": false, 2026-08-18 14:41:03.077784 | node1 | "failed": false, 2026-08-18 14:41:03.077805 | node1 | "invocation": { 2026-08-18 14:41:03.077824 | node1 | "module_args": { 2026-08-18 14:41:03.077855 | node1 | "checksum_algorithm": "sha1", 2026-08-18 14:41:03.077875 | node1 | "follow": true, 2026-08-18 14:41:03.077893 | node1 | "get_attributes": true, 2026-08-18 14:41:03.077913 | node1 | "get_checksum": true, 2026-08-18 14:41:03.077931 | node1 | "get_mime": true, 2026-08-18 14:41:03.077949 | node1 | "path": "src/github.com/vexxhost/ansible-collection-ceph" 2026-08-18 14:41:03.077968 | node1 | } 2026-08-18 14:41:03.077986 | node1 | }, 2026-08-18 14:41:03.078003 | node1 | "item": "src/github.com/vexxhost/ansible-collection-ceph", 2026-08-18 14:41:03.078020 | node1 | "stat": { 2026-08-18 14:41:03.078038 | node1 | "atime": 1787064054.6857934, 2026-08-18 14:41:03.078056 | node1 | "attr_flags": "e", 2026-08-18 14:41:03.078074 | node1 | "attributes": [ 2026-08-18 14:41:03.078092 | node1 | "extents" 2026-08-18 14:41:03.078109 | node1 | ], 2026-08-18 14:41:03.078126 | node1 | "block_size": 4096, 2026-08-18 14:41:03.078144 | node1 | "blocks": 8, 2026-08-18 14:41:03.078161 | node1 | "charset": "unknown", 2026-08-18 14:41:03.078179 | node1 | "ctime": 1787064054.700796, 2026-08-18 14:41:03.078197 | node1 | "dev": 64769, 2026-08-18 14:41:03.078215 | node1 | "device_type": 0, 2026-08-18 14:41:03.078233 | node1 | "executable": true, 2026-08-18 14:41:03.078250 | node1 | "exists": true, 2026-08-18 14:41:03.078267 | node1 | "gid": 1000, 2026-08-18 14:41:03.078284 | node1 | "gr_name": "zuul", 2026-08-18 14:41:03.078302 | node1 | "inode": 1034272, 2026-08-18 14:41:03.078319 | node1 | "isblk": false, 2026-08-18 14:41:03.078337 | node1 | "ischr": false, 2026-08-18 14:41:03.078354 | node1 | "isdir": true, 2026-08-18 14:41:03.078372 | node1 | "isfifo": false, 2026-08-18 14:41:03.078389 | node1 | "isgid": false, 2026-08-18 14:41:03.078406 | node1 | "islnk": false, 2026-08-18 14:41:03.078423 | node1 | "isreg": false, 2026-08-18 14:41:03.078440 | node1 | "issock": false, 2026-08-18 14:41:03.078457 | node1 | "isuid": false, 2026-08-18 14:41:03.078474 | node1 | "mimetype": "unknown", 2026-08-18 14:41:03.078491 | node1 | "mode": "0755", 2026-08-18 14:41:03.078508 | node1 | "mtime": 1787064036.7023783, 2026-08-18 14:41:03.078525 | node1 | "nlink": 10, 2026-08-18 14:41:03.078542 | node1 | "path": "src/github.com/vexxhost/ansible-collection-ceph", 2026-08-18 14:41:03.078560 | node1 | "pw_name": "zuul", 2026-08-18 14:41:03.078578 | node1 | "readable": true, 2026-08-18 14:41:03.078596 | node1 | "rgrp": true, 2026-08-18 14:41:03.078614 | node1 | "roth": true, 2026-08-18 14:41:03.078632 | node1 | "rusr": true, 2026-08-18 14:41:03.078674 | node1 | "size": 4096, 2026-08-18 14:41:03.078694 | node1 | "uid": 1000, 2026-08-18 14:41:03.078712 | node1 | "version": "1263741697", 2026-08-18 14:41:03.078730 | node1 | "wgrp": false, 2026-08-18 14:41:03.078749 | node1 | "woth": false, 2026-08-18 14:41:03.078767 | node1 | "writeable": true, 2026-08-18 14:41:03.078785 | node1 | "wusr": true, 2026-08-18 14:41:03.078803 | node1 | "xgrp": true, 2026-08-18 14:41:03.078820 | node1 | "xoth": true, 2026-08-18 14:41:03.078838 | node1 | "xusr": true 2026-08-18 14:41:03.078856 | node1 | } 2026-08-18 14:41:03.078874 | node1 | } 2026-08-18 14:41:03.078910 | node1 | ok: All items complete 2026-08-18 14:41:03.078931 | 2026-08-18 14:41:03.080148 | node1 | skipping: Conditional result was False 2026-08-18 14:41:03.080725 | node1 | skipping: Conditional result was False 2026-08-18 14:41:03.101013 | 2026-08-18 14:41:03.101190 | LOOP [Switch "ansible_host" to private IP] 2026-08-18 14:41:03.479241 | node1 | changed: 3 replacements made 2026-08-18 14:41:03.480850 | node1 | skipping: Conditional result was False 2026-08-18 14:41:03.481675 | node1 | skipping: Conditional result was False 2026-08-18 14:41:03.501350 | 2026-08-18 14:41:03.501550 | TASK [Run Molecule scenario] 2026-08-18 14:41:04.019577 | node1 | Using CPython 3.12.3 interpreter at: /usr/bin/python3 2026-08-18 14:41:04.019749 | node1 | Creating virtual environment at: .venv 2026-08-18 14:41:04.064152 | node1 | Downloading pygments (1.2MiB) 2026-08-18 14:41:04.064473 | node1 | Downloading cryptography (4.3MiB) 2026-08-18 14:41:04.066550 | node1 | Downloading netaddr (2.2MiB) 2026-08-18 14:41:04.067879 | node1 | Downloading ansible-core (2.1MiB) 2026-08-18 14:41:04.279324 | node1 | Downloading netaddr 2026-08-18 14:41:04.289733 | node1 | Downloading cryptography 2026-08-18 14:41:04.341408 | node1 | Downloading pygments 2026-08-18 14:41:04.485267 | node1 | Downloading ansible-core 2026-08-18 14:41:04.531273 | node1 | Installed 43 packages in 45ms 2026-08-18 14:41:05.282975 | node1 | INFO Collection 'vexxhost.ceph' detected. 2026-08-18 14:41:05.283125 | node1 | INFO Scenarios will be used from 'extensions/molecule' 2026-08-18 14:41:05.905258 | node1 | INFO [default > discovery] scenario test matrix: dependency, cleanup, destroy, syntax, create, prepare, converge, idempotence, side_effect, verify, cleanup, destroy 2026-08-18 14:41:05.905348 | node1 | INFO [default > prerun] Performing prerun with role_name_check=0... 2026-08-18 14:41:33.404750 | node1 | INFO [default > dependency] Executing 2026-08-18 14:41:33.404939 | node1 | WARNING [default > dependency] Missing roles requirements file: requirements.yml 2026-08-18 14:41:33.405305 | node1 | WARNING [default > dependency] Missing collections requirements file: collections.yml 2026-08-18 14:41:33.405415 | node1 | WARNING [default > dependency] Executed: 2 missing (Remove from test_sequence to suppress) 2026-08-18 14:41:33.411560 | node1 | INFO [default > cleanup] Executing 2026-08-18 14:41:33.411811 | node1 | WARNING [default > cleanup] Executed: Missing playbook (Remove from test_sequence to suppress) 2026-08-18 14:41:33.417407 | node1 | INFO [default > destroy] Executing 2026-08-18 14:41:33.417438 | node1 | WARNING [default > destroy] Skipping, '--destroy=never' requested. 2026-08-18 14:41:33.417500 | node1 | INFO [default > destroy] Executed: Successful 2026-08-18 14:41:33.423075 | node1 | INFO [default > syntax] Executing 2026-08-18 14:41:34.713703 | node1 | 2026-08-18 14:41:34.713970 | node1 | playbook: /home/zuul/src/github.com/vexxhost/ansible-collection-ceph/extensions/molecule/default/converge.yml 2026-08-18 14:41:34.775470 | node1 | INFO [default > syntax] Executed: Successful 2026-08-18 14:41:34.785266 | node1 | INFO [default > create] Executing 2026-08-18 14:41:34.787297 | node1 | WARNING [default > create] Executed: Missing playbook (Remove from test_sequence to suppress) 2026-08-18 14:41:34.793525 | node1 | INFO [default > prepare] Executing 2026-08-18 14:41:35.595194 | node1 | 2026-08-18 14:41:35.595358 | node1 | PLAY [Prepare] ***************************************************************** 2026-08-18 14:41:35.595583 | node1 | 2026-08-18 14:41:35.595829 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:41:35.596119 | node1 | Tuesday 18 August 2026 14:41:35 +0000 (0:00:00.032) 0:00:00.032 ******** 2026-08-18 14:41:36.898175 | node1 | [WARNING]: Platform linux on host node1 is using the discovered Python 2026-08-18 14:41:36.898383 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 14:41:36.898628 | node1 | interpreter could change the meaning of that path. See 2026-08-18 14:41:36.898866 | node1 | https://docs.ansible.com/ansible- 2026-08-18 14:41:36.899172 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 14:41:36.902808 | node1 | [WARNING]: Platform linux on host node2 is using the discovered Python 2026-08-18 14:41:36.903086 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 14:41:36.903395 | node1 | interpreter could change the meaning of that path. See 2026-08-18 14:41:36.903634 | node1 | https://docs.ansible.com/ansible- 2026-08-18 14:41:36.903879 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 14:41:36.918317 | node1 | [WARNING]: Platform linux on host node3 is using the discovered Python 2026-08-18 14:41:36.918520 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 14:41:36.918761 | node1 | interpreter could change the meaning of that path. See 2026-08-18 14:41:36.919000 | node1 | https://docs.ansible.com/ansible- 2026-08-18 14:41:36.919288 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 14:41:36.930264 | node1 | ok: [node1] 2026-08-18 14:41:36.930474 | node1 | ok: [node2] 2026-08-18 14:41:36.930707 | node1 | ok: [node3] 2026-08-18 14:41:36.930928 | node1 | 2026-08-18 14:41:36.931235 | node1 | TASK [Configure short hostname] ************************************************ 2026-08-18 14:41:36.931475 | node1 | Tuesday 18 August 2026 14:41:36 +0000 (0:00:01.335) 0:00:01.367 ******** 2026-08-18 14:41:37.718524 | node1 | changed: [node1] 2026-08-18 14:41:37.718752 | node1 | changed: [node2] 2026-08-18 14:41:37.719019 | node1 | changed: [node3] 2026-08-18 14:41:37.719293 | node1 | 2026-08-18 14:41:37.719572 | node1 | TASK [Ensure hostname inside hosts file] *************************************** 2026-08-18 14:41:37.719881 | node1 | Tuesday 18 August 2026 14:41:37 +0000 (0:00:00.787) 0:00:02.155 ******** 2026-08-18 14:41:38.045769 | node1 | [WARNING]: Module remote_tmp /root/.ansible/tmp did not exist and was created 2026-08-18 14:41:38.045851 | node1 | with a mode of 0700, this may cause issues when running as another user. To 2026-08-18 14:41:38.045943 | node1 | avoid this, create the remote_tmp dir with the correct permissions manually 2026-08-18 14:41:38.066957 | node1 | changed: [node2] 2026-08-18 14:41:38.067059 | node1 | changed: [node1] 2026-08-18 14:41:38.067170 | node1 | changed: [node3] 2026-08-18 14:41:38.067271 | node1 | 2026-08-18 14:41:38.067381 | node1 | TASK [Update apt cache] ******************************************************** 2026-08-18 14:41:38.067493 | node1 | Tuesday 18 August 2026 14:41:38 +0000 (0:00:00.349) 0:00:02.505 ******** 2026-08-18 14:41:52.490499 | node1 | changed: [node3] 2026-08-18 14:41:52.490674 | node1 | changed: [node2] 2026-08-18 14:41:52.490945 | node1 | changed: [node1] 2026-08-18 14:41:52.491228 | node1 | 2026-08-18 14:41:52.491472 | node1 | PLAY [Create devices for Ceph] ************************************************* 2026-08-18 14:41:52.491699 | node1 | 2026-08-18 14:41:52.491942 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:41:52.492234 | node1 | Tuesday 18 August 2026 14:41:52 +0000 (0:00:14.423) 0:00:16.928 ******** 2026-08-18 14:41:53.335473 | node1 | ok: [node1] 2026-08-18 14:41:53.335701 | node1 | ok: [node2] 2026-08-18 14:41:53.336069 | node1 | ok: [node3] 2026-08-18 14:41:53.336303 | node1 | 2026-08-18 14:41:53.336557 | node1 | TASK [Install depedencies] ***************************************************** 2026-08-18 14:41:53.336842 | node1 | Tuesday 18 August 2026 14:41:53 +0000 (0:00:00.844) 0:00:17.772 ******** 2026-08-18 14:42:16.628410 | node1 | changed: [node2] 2026-08-18 14:42:16.628624 | node1 | changed: [node3] 2026-08-18 14:42:16.628886 | node1 | changed: [node1] 2026-08-18 14:42:16.629154 | node1 | 2026-08-18 14:42:16.629410 | node1 | TASK [Start up service] ******************************************************** 2026-08-18 14:42:16.629664 | node1 | Tuesday 18 August 2026 14:42:16 +0000 (0:00:23.292) 0:00:41.065 ******** 2026-08-18 14:42:17.295265 | node1 | ok: [node2] 2026-08-18 14:42:17.295389 | node1 | ok: [node1] 2026-08-18 14:42:17.295402 | node1 | ok: [node3] 2026-08-18 14:42:17.295415 | node1 | 2026-08-18 14:42:17.295437 | node1 | TASK [Generate lvm.conf] ******************************************************* 2026-08-18 14:42:17.295450 | node1 | Tuesday 18 August 2026 14:42:17 +0000 (0:00:00.663) 0:00:41.729 ******** 2026-08-18 14:42:17.698158 | node1 | ok: [node2] 2026-08-18 14:42:17.698246 | node1 | ok: [node1] 2026-08-18 14:42:17.698258 | node1 | ok: [node3] 2026-08-18 14:42:17.698277 | node1 | 2026-08-18 14:42:17.698374 | node1 | TASK [Write /etc/lvm/lvm.conf] ************************************************* 2026-08-18 14:42:17.698533 | node1 | Tuesday 18 August 2026 14:42:17 +0000 (0:00:00.406) 0:00:42.135 ******** 2026-08-18 14:42:18.453242 | node1 | changed: [node1] 2026-08-18 14:42:18.453457 | node1 | changed: [node2] 2026-08-18 14:42:18.453695 | node1 | changed: [node3] 2026-08-18 14:42:18.453914 | node1 | 2026-08-18 14:42:18.454210 | node1 | TASK [Get list of all loopback devices] **************************************** 2026-08-18 14:42:18.454464 | node1 | Tuesday 18 August 2026 14:42:18 +0000 (0:00:00.755) 0:00:42.890 ******** 2026-08-18 14:42:18.725053 | node1 | ok: [node2] 2026-08-18 14:42:18.725086 | node1 | ok: [node1] 2026-08-18 14:42:18.726361 | node1 | ok: [node3] 2026-08-18 14:42:18.726696 | node1 | 2026-08-18 14:42:18.726955 | node1 | TASK [Fail if there is any existing loopback devices] ************************** 2026-08-18 14:42:18.727247 | node1 | Tuesday 18 August 2026 14:42:18 +0000 (0:00:00.271) 0:00:43.161 ******** 2026-08-18 14:42:18.775809 | node1 | skipping: [node1] 2026-08-18 14:42:18.776083 | node1 | skipping: [node2] 2026-08-18 14:42:18.776323 | node1 | skipping: [node3] 2026-08-18 14:42:18.776542 | node1 | 2026-08-18 14:42:18.776843 | node1 | TASK [Create devices for Ceph] ************************************************* 2026-08-18 14:42:18.777191 | node1 | Tuesday 18 August 2026 14:42:18 +0000 (0:00:00.051) 0:00:43.213 ******** 2026-08-18 14:42:19.501705 | node1 | changed: [node2] => (item=osd0) 2026-08-18 14:42:19.501950 | node1 | changed: [node3] => (item=osd0) 2026-08-18 14:42:19.502250 | node1 | changed: [node1] => (item=osd0) 2026-08-18 14:42:19.502538 | node1 | changed: [node2] => (item=osd1) 2026-08-18 14:42:19.502798 | node1 | changed: [node3] => (item=osd1) 2026-08-18 14:42:19.503081 | node1 | changed: [node1] => (item=osd1) 2026-08-18 14:42:19.503331 | node1 | changed: [node2] => (item=osd2) 2026-08-18 14:42:19.503604 | node1 | changed: [node3] => (item=osd2) 2026-08-18 14:42:19.503849 | node1 | changed: [node1] => (item=osd2) 2026-08-18 14:42:19.504126 | node1 | 2026-08-18 14:42:19.504387 | node1 | TASK [Set permissions on loopback devices] ************************************* 2026-08-18 14:42:19.504662 | node1 | Tuesday 18 August 2026 14:42:19 +0000 (0:00:00.725) 0:00:43.939 ******** 2026-08-18 14:42:20.290378 | node1 | changed: [node2] => (item=osd0) 2026-08-18 14:42:20.290587 | node1 | changed: [node3] => (item=osd0) 2026-08-18 14:42:20.290836 | node1 | changed: [node1] => (item=osd0) 2026-08-18 14:42:20.291140 | node1 | changed: [node2] => (item=osd1) 2026-08-18 14:42:20.291388 | node1 | changed: [node3] => (item=osd1) 2026-08-18 14:42:20.291630 | node1 | changed: [node1] => (item=osd1) 2026-08-18 14:42:20.291900 | node1 | changed: [node2] => (item=osd2) 2026-08-18 14:42:20.292223 | node1 | changed: [node3] => (item=osd2) 2026-08-18 14:42:20.292464 | node1 | changed: [node1] => (item=osd2) 2026-08-18 14:42:20.292686 | node1 | 2026-08-18 14:42:20.292932 | node1 | TASK [Start loop devices] ****************************************************** 2026-08-18 14:42:20.293254 | node1 | Tuesday 18 August 2026 14:42:20 +0000 (0:00:00.788) 0:00:44.727 ******** 2026-08-18 14:42:21.133481 | node1 | changed: [node1] => (item=osd0) 2026-08-18 14:42:21.133709 | node1 | changed: [node3] => (item=osd0) 2026-08-18 14:42:21.134083 | node1 | changed: [node2] => (item=osd0) 2026-08-18 14:42:21.134342 | node1 | changed: [node2] => (item=osd1) 2026-08-18 14:42:21.134590 | node1 | changed: [node1] => (item=osd1) 2026-08-18 14:42:21.134832 | node1 | changed: [node3] => (item=osd1) 2026-08-18 14:42:21.135104 | node1 | changed: [node2] => (item=osd2) 2026-08-18 14:42:21.135347 | node1 | changed: [node1] => (item=osd2) 2026-08-18 14:42:21.135586 | node1 | changed: [node3] => (item=osd2) 2026-08-18 14:42:21.135805 | node1 | 2026-08-18 14:42:21.136082 | node1 | TASK [Create a volume group for each loop device] ****************************** 2026-08-18 14:42:21.136334 | node1 | Tuesday 18 August 2026 14:42:21 +0000 (0:00:00.843) 0:00:45.571 ******** 2026-08-18 14:42:22.572867 | node1 | changed: [node2] => (item=osd0) 2026-08-18 14:42:22.575000 | node1 | changed: [node3] => (item=osd0) 2026-08-18 14:42:22.575055 | node1 | changed: [node1] => (item=osd0) 2026-08-18 14:42:22.575059 | node1 | changed: [node2] => (item=osd1) 2026-08-18 14:42:22.575063 | node1 | changed: [node3] => (item=osd1) 2026-08-18 14:42:22.575067 | node1 | changed: [node1] => (item=osd1) 2026-08-18 14:42:22.575070 | node1 | changed: [node2] => (item=osd2) 2026-08-18 14:42:22.575074 | node1 | changed: [node3] => (item=osd2) 2026-08-18 14:42:22.575078 | node1 | changed: [node1] => (item=osd2) 2026-08-18 14:42:22.575082 | node1 | 2026-08-18 14:42:22.575086 | node1 | TASK [Create a logical volume for each loop device] **************************** 2026-08-18 14:42:22.575091 | node1 | Tuesday 18 August 2026 14:42:22 +0000 (0:00:01.437) 0:00:47.008 ******** 2026-08-18 14:42:23.965616 | node1 | changed: [node2] => (item=ceph-node2-osd0) 2026-08-18 14:42:23.965878 | node1 | changed: [node3] => (item=ceph-node3-osd0) 2026-08-18 14:42:23.966186 | node1 | changed: [node1] => (item=ceph-node1-osd0) 2026-08-18 14:42:23.966472 | node1 | changed: [node2] => (item=ceph-node2-osd1) 2026-08-18 14:42:23.966748 | node1 | changed: [node3] => (item=ceph-node3-osd1) 2026-08-18 14:42:23.967035 | node1 | changed: [node1] => (item=ceph-node1-osd1) 2026-08-18 14:42:23.967355 | node1 | changed: [node2] => (item=ceph-node2-osd2) 2026-08-18 14:42:23.967691 | node1 | changed: [node3] => (item=ceph-node3-osd2) 2026-08-18 14:42:23.967968 | node1 | changed: [node1] => (item=ceph-node1-osd2) 2026-08-18 14:42:23.968266 | node1 | 2026-08-18 14:42:23.968642 | node1 | PLAY RECAP ********************************************************************* 2026-08-18 14:42:23.972033 | node1 | node1 : ok=15 changed=10 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0 2026-08-18 14:42:23.972051 | node1 | node2 : ok=15 changed=10 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0 2026-08-18 14:42:23.972056 | node1 | node3 : ok=15 changed=10 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0 2026-08-18 14:42:23.972062 | node1 | 2026-08-18 14:42:23.972067 | node1 | 2026-08-18 14:42:23.972073 | node1 | TASKS RECAP ******************************************************************** 2026-08-18 14:42:23.972079 | node1 | Tuesday 18 August 2026 14:42:23 +0000 (0:00:01.394) 0:00:48.403 ******** 2026-08-18 14:42:23.972084 | node1 | =============================================================================== 2026-08-18 14:42:23.972089 | node1 | Install depedencies ---------------------------------------------------- 23.29s 2026-08-18 14:42:23.972094 | node1 | Update apt cache ------------------------------------------------------- 14.42s 2026-08-18 14:42:23.972099 | node1 | Create a volume group for each loop device ------------------------------ 1.44s 2026-08-18 14:42:23.972105 | node1 | Create a logical volume for each loop device ---------------------------- 1.40s 2026-08-18 14:42:23.972110 | node1 | Gathering Facts --------------------------------------------------------- 1.34s 2026-08-18 14:42:23.972115 | node1 | Gathering Facts --------------------------------------------------------- 0.84s 2026-08-18 14:42:23.972123 | node1 | Start loop devices ------------------------------------------------------ 0.84s 2026-08-18 14:42:23.972289 | node1 | Set permissions on loopback devices ------------------------------------- 0.79s 2026-08-18 14:42:23.972443 | node1 | Configure short hostname ------------------------------------------------ 0.79s 2026-08-18 14:42:23.972590 | node1 | Write /etc/lvm/lvm.conf ------------------------------------------------- 0.76s 2026-08-18 14:42:23.972741 | node1 | Create devices for Ceph ------------------------------------------------- 0.73s 2026-08-18 14:42:23.972894 | node1 | Start up service -------------------------------------------------------- 0.66s 2026-08-18 14:42:23.973057 | node1 | Generate lvm.conf ------------------------------------------------------- 0.41s 2026-08-18 14:42:23.973208 | node1 | Ensure hostname inside hosts file --------------------------------------- 0.35s 2026-08-18 14:42:23.973361 | node1 | Get list of all loopback devices ---------------------------------------- 0.27s 2026-08-18 14:42:23.973509 | node1 | Fail if there is any existing loopback devices -------------------------- 0.05s 2026-08-18 14:42:24.059170 | node1 | INFO [default > prepare] Executed: Successful 2026-08-18 14:42:24.069267 | node1 | INFO [default > converge] Executing 2026-08-18 14:42:25.235322 | node1 | 2026-08-18 14:42:25.235523 | node1 | PLAY [Deploy Ceph monitors & managers] ***************************************** 2026-08-18 14:42:25.235753 | node1 | 2026-08-18 14:42:25.236088 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:42:25.236337 | node1 | Tuesday 18 August 2026 14:42:25 +0000 (0:00:00.058) 0:00:00.058 ******** 2026-08-18 14:42:26.465224 | node1 | [WARNING]: Platform linux on host node1 is using the discovered Python 2026-08-18 14:42:26.465296 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 14:42:26.465308 | node1 | interpreter could change the meaning of that path. See 2026-08-18 14:42:26.465318 | node1 | https://docs.ansible.com/ansible- 2026-08-18 14:42:26.465361 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 14:42:26.550887 | node1 | [WARNING]: Platform linux on host node2 is using the discovered Python 2026-08-18 14:42:26.551130 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 14:42:26.551367 | node1 | interpreter could change the meaning of that path. See 2026-08-18 14:42:26.551606 | node1 | https://docs.ansible.com/ansible- 2026-08-18 14:42:26.551846 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 14:42:27.530212 | node1 | [WARNING]: Platform linux on host node3 is using the discovered Python 2026-08-18 14:42:27.530443 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 14:42:27.530684 | node1 | interpreter could change the meaning of that path. See 2026-08-18 14:42:27.530923 | node1 | https://docs.ansible.com/ansible- 2026-08-18 14:42:27.531197 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 14:42:27.590438 | node1 | ok: [node1] 2026-08-18 14:42:27.590619 | node1 | ok: [node2] 2026-08-18 14:42:27.590851 | node1 | ok: [node3] 2026-08-18 14:42:27.591113 | node1 | 2026-08-18 14:42:27.591373 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:42:27.591596 | node1 | Tuesday 18 August 2026 14:42:27 +0000 (0:00:02.355) 0:00:02.413 ******** 2026-08-18 14:42:27.978691 | node1 | ok: [node1] 2026-08-18 14:42:27.978892 | node1 | ok: [node2] 2026-08-18 14:42:27.979161 | node1 | ok: [node3] 2026-08-18 14:42:27.979381 | node1 | 2026-08-18 14:42:27.979627 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:42:27.979869 | node1 | Tuesday 18 August 2026 14:42:27 +0000 (0:00:00.387) 0:00:02.801 ******** 2026-08-18 14:42:28.021406 | node1 | skipping: [node1] 2026-08-18 14:42:28.086139 | node1 | skipping: [node2] 2026-08-18 14:42:28.086361 | node1 | skipping: [node3] 2026-08-18 14:42:28.086577 | node1 | 2026-08-18 14:42:28.086826 | node1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-08-18 14:42:28.087111 | node1 | Tuesday 18 August 2026 14:42:28 +0000 (0:00:00.107) 0:00:02.909 ******** 2026-08-18 14:42:28.472903 | node1 | changed: [node1] 2026-08-18 14:42:28.473142 | node1 | changed: [node2] 2026-08-18 14:42:28.473378 | node1 | changed: [node3] 2026-08-18 14:42:28.473641 | node1 | 2026-08-18 14:42:28.473884 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:42:28.474190 | node1 | Tuesday 18 August 2026 14:42:28 +0000 (0:00:00.385) 0:00:03.294 ******** 2026-08-18 14:42:28.580445 | node1 | ok: [node1] => { 2026-08-18 14:42:28.580662 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:42:28.580900 | node1 | } 2026-08-18 14:42:28.581181 | node1 | ok: [node2] => { 2026-08-18 14:42:28.581432 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:42:28.581709 | node1 | } 2026-08-18 14:42:28.581924 | node1 | ok: [node3] => { 2026-08-18 14:42:28.582224 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:42:28.582439 | node1 | } 2026-08-18 14:42:28.582661 | node1 | 2026-08-18 14:42:28.583221 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:42:28.583477 | node1 | Tuesday 18 August 2026 14:42:28 +0000 (0:00:00.107) 0:00:03.402 ******** 2026-08-18 14:42:29.560187 | node1 | changed: [node1] 2026-08-18 14:42:29.560267 | node1 | changed: [node3] 2026-08-18 14:42:29.560434 | node1 | changed: [node2] 2026-08-18 14:42:29.560660 | node1 | 2026-08-18 14:42:29.560915 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:42:29.561208 | node1 | Tuesday 18 August 2026 14:42:29 +0000 (0:00:00.979) 0:00:04.382 ******** 2026-08-18 14:42:29.597691 | node1 | skipping: [node1] 2026-08-18 14:42:29.689207 | node1 | skipping: [node2] 2026-08-18 14:42:29.689420 | node1 | skipping: [node3] 2026-08-18 14:42:29.689648 | node1 | 2026-08-18 14:42:29.689909 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:42:29.690208 | node1 | Tuesday 18 August 2026 14:42:29 +0000 (0:00:00.129) 0:00:04.511 ******** 2026-08-18 14:42:29.810461 | node1 | skipping: [node1] 2026-08-18 14:42:29.810687 | node1 | skipping: [node2] 2026-08-18 14:42:29.810924 | node1 | skipping: [node3] 2026-08-18 14:42:29.811206 | node1 | 2026-08-18 14:42:29.811461 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:42:29.811710 | node1 | Tuesday 18 August 2026 14:42:29 +0000 (0:00:00.120) 0:00:04.632 ******** 2026-08-18 14:42:30.119664 | node1 | ok: [node1] 2026-08-18 14:42:30.119883 | node1 | ok: [node2] 2026-08-18 14:42:30.120161 | node1 | ok: [node3] 2026-08-18 14:42:30.120384 | node1 | 2026-08-18 14:42:30.120640 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:42:30.120891 | node1 | Tuesday 18 August 2026 14:42:30 +0000 (0:00:00.309) 0:00:04.942 ******** 2026-08-18 14:42:31.359213 | node1 | ok: [node2] 2026-08-18 14:42:31.359278 | node1 | ok: [node1] 2026-08-18 14:42:31.360315 | node1 | ok: [node3] 2026-08-18 14:42:31.360405 | node1 | 2026-08-18 14:42:31.360416 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:42:31.360426 | node1 | Tuesday 18 August 2026 14:42:31 +0000 (0:00:01.236) 0:00:06.178 ******** 2026-08-18 14:42:31.602091 | node1 | ok: [node1] => { 2026-08-18 14:42:31.602162 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:42:31.602173 | node1 | } 2026-08-18 14:42:31.602182 | node1 | ok: [node2] => { 2026-08-18 14:42:31.602190 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:42:31.602199 | node1 | } 2026-08-18 14:42:31.602207 | node1 | ok: [node3] => { 2026-08-18 14:42:31.602215 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:42:31.602224 | node1 | } 2026-08-18 14:42:31.602347 | node1 | 2026-08-18 14:42:31.602670 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:42:31.602977 | node1 | Tuesday 18 August 2026 14:42:31 +0000 (0:00:00.243) 0:00:06.421 ******** 2026-08-18 14:42:32.790629 | node1 | changed: [node1] 2026-08-18 14:42:32.790994 | node1 | changed: [node3] 2026-08-18 14:42:32.791353 | node1 | changed: [node2] 2026-08-18 14:42:32.791636 | node1 | 2026-08-18 14:42:32.791987 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:42:32.792321 | node1 | Tuesday 18 August 2026 14:42:32 +0000 (0:00:01.191) 0:00:07.613 ******** 2026-08-18 14:42:36.019271 | node1 | changed: [node1] 2026-08-18 14:42:36.019644 | node1 | changed: [node2] 2026-08-18 14:42:36.019948 | node1 | changed: [node3] 2026-08-18 14:42:36.020268 | node1 | 2026-08-18 14:42:36.020594 | node1 | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-08-18 14:42:36.020901 | node1 | Tuesday 18 August 2026 14:42:36 +0000 (0:00:03.228) 0:00:10.842 ******** 2026-08-18 14:42:36.059855 | node1 | skipping: [node1] 2026-08-18 14:42:36.087561 | node1 | skipping: [node2] 2026-08-18 14:42:36.087803 | node1 | skipping: [node3] 2026-08-18 14:42:36.087983 | node1 | 2026-08-18 14:42:36.088286 | node1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-08-18 14:42:36.088605 | node1 | Tuesday 18 August 2026 14:42:36 +0000 (0:00:00.068) 0:00:10.910 ******** 2026-08-18 14:42:36.155440 | node1 | skipping: [node1] 2026-08-18 14:42:36.155526 | node1 | skipping: [node2] 2026-08-18 14:42:36.155633 | node1 | skipping: [node3] 2026-08-18 14:42:36.155733 | node1 | 2026-08-18 14:42:36.155847 | node1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-08-18 14:42:36.155952 | node1 | Tuesday 18 August 2026 14:42:36 +0000 (0:00:00.067) 0:00:10.977 ******** 2026-08-18 14:42:36.192576 | node1 | skipping: [node1] 2026-08-18 14:42:36.228466 | node1 | skipping: [node2] 2026-08-18 14:42:36.228552 | node1 | skipping: [node3] 2026-08-18 14:42:36.228646 | node1 | 2026-08-18 14:42:36.228772 | node1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-08-18 14:42:36.228921 | node1 | Tuesday 18 August 2026 14:42:36 +0000 (0:00:00.073) 0:00:11.051 ******** 2026-08-18 14:42:44.126165 | node1 | changed: [node1] 2026-08-18 14:42:44.126244 | node1 | changed: [node2] 2026-08-18 14:42:44.126254 | node1 | changed: [node3] 2026-08-18 14:42:44.126263 | node1 | 2026-08-18 14:42:44.126273 | node1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-08-18 14:42:44.126291 | node1 | Tuesday 18 August 2026 14:42:44 +0000 (0:00:07.894) 0:00:18.946 ******** 2026-08-18 14:42:44.792794 | node1 | changed: [node1] 2026-08-18 14:42:44.792839 | node1 | changed: [node3] 2026-08-18 14:42:44.792848 | node1 | changed: [node2] 2026-08-18 14:42:44.792858 | node1 | 2026-08-18 14:42:44.792867 | node1 | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-08-18 14:42:44.792876 | node1 | Tuesday 18 August 2026 14:42:44 +0000 (0:00:00.668) 0:00:19.614 ******** 2026-08-18 14:42:46.005095 | node1 | changed: [node1] => (item={'path': '/etc/containerd'}) 2026-08-18 14:42:46.005150 | node1 | changed: [node2] => (item={'path': '/etc/containerd'}) 2026-08-18 14:42:46.005160 | node1 | changed: [node3] => (item={'path': '/etc/containerd'}) 2026-08-18 14:42:46.005169 | node1 | changed: [node1] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:42:46.005178 | node1 | changed: [node2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:42:46.005186 | node1 | changed: [node3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:42:46.005194 | node1 | changed: [node1] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:42:46.005203 | node1 | changed: [node2] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:42:46.005211 | node1 | changed: [node3] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:42:46.005219 | node1 | changed: [node1] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:42:46.005260 | node1 | changed: [node2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:42:46.005588 | node1 | changed: [node3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:42:46.005912 | node1 | changed: [node1] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:42:46.006278 | node1 | changed: [node2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:42:46.006702 | node1 | changed: [node3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:42:46.006994 | node1 | 2026-08-18 14:42:46.007345 | node1 | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-08-18 14:42:46.007545 | node1 | Tuesday 18 August 2026 14:42:46 +0000 (0:00:01.210) 0:00:20.825 ******** 2026-08-18 14:42:46.822774 | node1 | changed: [node1] 2026-08-18 14:42:46.823272 | node1 | changed: [node2] 2026-08-18 14:42:46.823609 | node1 | changed: [node3] 2026-08-18 14:42:46.823927 | node1 | 2026-08-18 14:42:46.824322 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:42:46.824661 | node1 | Tuesday 18 August 2026 14:42:46 +0000 (0:00:00.796) 0:00:21.621 ******** 2026-08-18 14:42:46.824970 | node1 | 2026-08-18 14:42:46.825358 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:42:46.825688 | node1 | Tuesday 18 August 2026 14:42:46 +0000 (0:00:00.008) 0:00:21.629 ******** 2026-08-18 14:42:46.825979 | node1 | 2026-08-18 14:42:46.826354 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:42:46.826681 | node1 | Tuesday 18 August 2026 14:42:46 +0000 (0:00:00.007) 0:00:21.637 ******** 2026-08-18 14:42:46.826970 | node1 | 2026-08-18 14:42:46.827391 | node1 | RUNNING HANDLER [vexxhost.containers.containerd : Reload systemd] ************** 2026-08-18 14:42:46.827800 | node1 | Tuesday 18 August 2026 14:42:46 +0000 (0:00:00.007) 0:00:21.645 ******** 2026-08-18 14:42:47.993660 | node1 | ok: [node2] 2026-08-18 14:42:47.994073 | node1 | ok: [node1] 2026-08-18 14:42:47.994401 | node1 | ok: [node3] 2026-08-18 14:42:47.994691 | node1 | 2026-08-18 14:42:47.995017 | node1 | RUNNING HANDLER [vexxhost.containers.containerd : Restart containerd] ********** 2026-08-18 14:42:47.995372 | node1 | Tuesday 18 August 2026 14:42:47 +0000 (0:00:01.171) 0:00:22.816 ******** 2026-08-18 14:42:48.604722 | node1 | changed: [node2] 2026-08-18 14:42:48.605273 | node1 | changed: [node1] 2026-08-18 14:42:48.605599 | node1 | changed: [node3] 2026-08-18 14:42:48.605882 | node1 | 2026-08-18 14:42:48.606238 | node1 | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-08-18 14:42:48.606558 | node1 | Tuesday 18 August 2026 14:42:48 +0000 (0:00:00.611) 0:00:23.427 ******** 2026-08-18 14:42:49.361987 | node1 | changed: [node2] 2026-08-18 14:42:49.363113 | node1 | changed: [node1] 2026-08-18 14:42:49.363150 | node1 | changed: [node3] 2026-08-18 14:42:49.363171 | node1 | 2026-08-18 14:42:49.363800 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:42:49.363823 | node1 | Tuesday 18 August 2026 14:42:49 +0000 (0:00:00.757) 0:00:24.184 ******** 2026-08-18 14:42:49.665949 | node1 | ok: [node1] 2026-08-18 14:42:49.665981 | node1 | ok: [node2] 2026-08-18 14:42:49.665986 | node1 | ok: [node3] 2026-08-18 14:42:49.665990 | node1 | 2026-08-18 14:42:49.665994 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:42:49.665998 | node1 | Tuesday 18 August 2026 14:42:49 +0000 (0:00:00.303) 0:00:24.487 ******** 2026-08-18 14:42:49.852262 | node1 | ok: [node1] => { 2026-08-18 14:42:49.852361 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:42:49.852680 | node1 | } 2026-08-18 14:42:49.852995 | node1 | ok: [node2] => { 2026-08-18 14:42:49.853380 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:42:49.853697 | node1 | } 2026-08-18 14:42:49.854075 | node1 | ok: [node3] => { 2026-08-18 14:42:49.855077 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:42:49.855105 | node1 | } 2026-08-18 14:42:49.855115 | node1 | 2026-08-18 14:42:49.855386 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:42:49.855702 | node1 | Tuesday 18 August 2026 14:42:49 +0000 (0:00:00.186) 0:00:24.674 ******** 2026-08-18 14:42:51.016879 | node1 | changed: [node2] 2026-08-18 14:42:51.017286 | node1 | changed: [node3] 2026-08-18 14:42:51.017602 | node1 | changed: [node1] 2026-08-18 14:42:51.017888 | node1 | 2026-08-18 14:42:51.018241 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:42:51.018564 | node1 | Tuesday 18 August 2026 14:42:51 +0000 (0:00:01.164) 0:00:25.838 ******** 2026-08-18 14:42:55.844808 | node1 | changed: [node2] 2026-08-18 14:42:55.846238 | node1 | changed: [node3] 2026-08-18 14:42:55.846267 | node1 | changed: [node1] 2026-08-18 14:42:55.846280 | node1 | 2026-08-18 14:42:55.846292 | node1 | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-08-18 14:42:55.846305 | node1 | Tuesday 18 August 2026 14:42:55 +0000 (0:00:04.828) 0:00:30.666 ******** 2026-08-18 14:42:57.062074 | node1 | ok: [node1] 2026-08-18 14:42:57.062113 | node1 | ok: [node2] 2026-08-18 14:42:57.062121 | node1 | ok: [node3] 2026-08-18 14:42:57.062129 | node1 | 2026-08-18 14:42:57.062137 | node1 | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-08-18 14:42:57.062149 | node1 | Tuesday 18 August 2026 14:42:57 +0000 (0:00:01.216) 0:00:31.883 ******** 2026-08-18 14:42:57.473454 | node1 | changed: [node3] 2026-08-18 14:42:57.473515 | node1 | changed: [node1] 2026-08-18 14:42:57.473610 | node1 | changed: [node2] 2026-08-18 14:42:57.473705 | node1 | 2026-08-18 14:42:57.473819 | node1 | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-08-18 14:42:57.473941 | node1 | Tuesday 18 August 2026 14:42:57 +0000 (0:00:00.410) 0:00:32.293 ******** 2026-08-18 14:42:58.078202 | node1 | changed: [node1] 2026-08-18 14:42:58.078264 | node1 | changed: [node2] 2026-08-18 14:42:58.078354 | node1 | changed: [node3] 2026-08-18 14:42:58.078466 | node1 | 2026-08-18 14:42:58.078635 | node1 | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-08-18 14:42:58.081157 | node1 | Tuesday 18 August 2026 14:42:58 +0000 (0:00:00.607) 0:00:32.901 ******** 2026-08-18 14:42:58.815139 | node1 | changed: [node1] => (item={'path': '/etc/docker'}) 2026-08-18 14:42:58.815194 | node1 | changed: [node2] => (item={'path': '/etc/docker'}) 2026-08-18 14:42:58.817074 | node1 | changed: [node3] => (item={'path': '/etc/docker'}) 2026-08-18 14:42:58.817132 | node1 | changed: [node1] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:42:58.817142 | node1 | changed: [node2] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:42:58.817151 | node1 | changed: [node3] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:42:58.817159 | node1 | changed: [node1] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:42:58.817168 | node1 | changed: [node2] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:42:58.817176 | node1 | changed: [node3] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:42:58.817453 | node1 | 2026-08-18 14:42:58.817838 | node1 | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-08-18 14:42:58.818349 | node1 | Tuesday 18 August 2026 14:42:58 +0000 (0:00:00.735) 0:00:33.636 ******** 2026-08-18 14:42:59.420432 | node1 | changed: [node1] 2026-08-18 14:42:59.420493 | node1 | changed: [node2] 2026-08-18 14:42:59.420613 | node1 | changed: [node3] 2026-08-18 14:42:59.420708 | node1 | 2026-08-18 14:42:59.420823 | node1 | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-08-18 14:42:59.420933 | node1 | Tuesday 18 August 2026 14:42:59 +0000 (0:00:00.606) 0:00:34.242 ******** 2026-08-18 14:43:00.059472 | node1 | changed: [node1] 2026-08-18 14:43:00.059525 | node1 | changed: [node2] 2026-08-18 14:43:00.059617 | node1 | changed: [node3] 2026-08-18 14:43:00.059714 | node1 | 2026-08-18 14:43:00.059828 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:43:00.059940 | node1 | Tuesday 18 August 2026 14:43:00 +0000 (0:00:00.613) 0:00:34.855 ******** 2026-08-18 14:43:00.060059 | node1 | 2026-08-18 14:43:00.060180 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:43:00.060290 | node1 | Tuesday 18 August 2026 14:43:00 +0000 (0:00:00.008) 0:00:34.864 ******** 2026-08-18 14:43:00.060389 | node1 | 2026-08-18 14:43:00.060499 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:43:00.060608 | node1 | Tuesday 18 August 2026 14:43:00 +0000 (0:00:00.008) 0:00:34.872 ******** 2026-08-18 14:43:00.060707 | node1 | 2026-08-18 14:43:00.060833 | node1 | RUNNING HANDLER [vexxhost.containers.containerd : Reload systemd] ************** 2026-08-18 14:43:00.060936 | node1 | Tuesday 18 August 2026 14:43:00 +0000 (0:00:00.009) 0:00:34.881 ******** 2026-08-18 14:43:01.022142 | node1 | ok: [node1] 2026-08-18 14:43:01.022231 | node1 | ok: [node2] 2026-08-18 14:43:01.022242 | node1 | ok: [node3] 2026-08-18 14:43:01.022252 | node1 | 2026-08-18 14:43:01.022263 | node1 | RUNNING HANDLER [vexxhost.containers.docker : Restart docker] ****************** 2026-08-18 14:43:01.022298 | node1 | Tuesday 18 August 2026 14:43:01 +0000 (0:00:00.960) 0:00:35.842 ******** 2026-08-18 14:43:02.425118 | node1 | changed: [node1] 2026-08-18 14:43:02.425190 | node1 | changed: [node3] 2026-08-18 14:43:02.426132 | node1 | changed: [node2] 2026-08-18 14:43:02.426167 | node1 | 2026-08-18 14:43:02.426181 | node1 | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-08-18 14:43:02.426193 | node1 | Tuesday 18 August 2026 14:43:02 +0000 (0:00:01.404) 0:00:37.247 ******** 2026-08-18 14:43:03.204206 | node1 | changed: [node1] 2026-08-18 14:43:03.204264 | node1 | changed: [node2] 2026-08-18 14:43:03.204277 | node1 | changed: [node3] 2026-08-18 14:43:03.204294 | node1 | 2026-08-18 14:43:03.204887 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:43:03.204920 | node1 | Tuesday 18 August 2026 14:43:03 +0000 (0:00:00.777) 0:00:38.024 ******** 2026-08-18 14:43:03.277476 | node1 | ok: [node1] => { 2026-08-18 14:43:03.277565 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:43:03.277664 | node1 | } 2026-08-18 14:43:03.277777 | node1 | ok: [node2] => { 2026-08-18 14:43:03.277890 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:43:03.277989 | node1 | } 2026-08-18 14:43:03.278122 | node1 | ok: [node3] => { 2026-08-18 14:43:03.279068 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:43:03.279119 | node1 | } 2026-08-18 14:43:03.279125 | node1 | 2026-08-18 14:43:03.279129 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:43:03.279134 | node1 | Tuesday 18 August 2026 14:43:03 +0000 (0:00:00.074) 0:00:38.099 ******** 2026-08-18 14:43:04.790457 | node1 | changed: [node2] 2026-08-18 14:43:04.790672 | node1 | changed: [node1] 2026-08-18 14:43:04.791002 | node1 | changed: [node3] 2026-08-18 14:43:04.791317 | node1 | 2026-08-18 14:43:04.791604 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:43:04.791859 | node1 | Tuesday 18 August 2026 14:43:04 +0000 (0:00:01.512) 0:00:39.612 ******** 2026-08-18 14:43:04.826433 | node1 | skipping: [node1] 2026-08-18 14:43:04.887983 | node1 | skipping: [node2] 2026-08-18 14:43:04.888273 | node1 | skipping: [node3] 2026-08-18 14:43:04.888658 | node1 | 2026-08-18 14:43:04.889142 | node1 | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-08-18 14:43:04.889429 | node1 | Tuesday 18 August 2026 14:43:04 +0000 (0:00:00.098) 0:00:39.710 ******** 2026-08-18 14:43:04.973789 | node1 | ok: [node1] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:43:04.974084 | node1 | ok: [node2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:43:04.974411 | node1 | ok: [node3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:43:04.974800 | node1 | 2026-08-18 14:43:04.975093 | node1 | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-08-18 14:43:04.975405 | node1 | Tuesday 18 August 2026 14:43:04 +0000 (0:00:00.086) 0:00:39.796 ******** 2026-08-18 14:43:06.153451 | node1 | ok: [node1] 2026-08-18 14:43:06.153574 | node1 | ok: [node2] 2026-08-18 14:43:06.153736 | node1 | ok: [node3] 2026-08-18 14:43:06.153982 | node1 | 2026-08-18 14:43:06.154280 | node1 | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-08-18 14:43:06.154558 | node1 | Tuesday 18 August 2026 14:43:06 +0000 (0:00:01.179) 0:00:40.976 ******** 2026-08-18 14:43:06.992418 | node1 | ok: [node1] => (item=systemd-timesyncd) 2026-08-18 14:43:06.992506 | node1 | ok: [node2] => (item=systemd-timesyncd) 2026-08-18 14:43:06.992621 | node1 | ok: [node3] => (item=systemd-timesyncd) 2026-08-18 14:43:06.992735 | node1 | ok: [node1] => (item=ssh) 2026-08-18 14:43:06.994101 | node1 | ok: [node2] => (item=ssh) 2026-08-18 14:43:06.994152 | node1 | ok: [node3] => (item=ssh) 2026-08-18 14:43:06.994157 | node1 | 2026-08-18 14:43:06.994161 | node1 | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-08-18 14:43:06.994166 | node1 | Tuesday 18 August 2026 14:43:06 +0000 (0:00:00.838) 0:00:41.814 ******** 2026-08-18 14:43:07.268165 | node1 | ok: [node1] 2026-08-18 14:43:07.268231 | node1 | ok: [node2] 2026-08-18 14:43:07.268241 | node1 | ok: [node3] 2026-08-18 14:43:07.268250 | node1 | 2026-08-18 14:43:07.268259 | node1 | TASK [vexxhost.ceph.cephadm : Ensure the cephadm SSH user exists] ************** 2026-08-18 14:43:07.268275 | node1 | Tuesday 18 August 2026 14:43:07 +0000 (0:00:00.274) 0:00:42.089 ******** 2026-08-18 14:43:07.802697 | node1 | changed: [node1] 2026-08-18 14:43:07.802893 | node1 | changed: [node2] 2026-08-18 14:43:07.803176 | node1 | changed: [node3] 2026-08-18 14:43:07.803392 | node1 | 2026-08-18 14:43:07.803681 | node1 | TASK [vexxhost.ceph.cephadm : Grant passwordless sudo to the cephadm SSH user] *** 2026-08-18 14:43:07.804051 | node1 | Tuesday 18 August 2026 14:43:07 +0000 (0:00:00.536) 0:00:42.625 ******** 2026-08-18 14:43:08.414642 | node1 | changed: [node1] 2026-08-18 14:43:08.414714 | node1 | changed: [node2] 2026-08-18 14:43:08.414786 | node1 | changed: [node3] 2026-08-18 14:43:08.414884 | node1 | 2026-08-18 14:43:08.415187 | node1 | TASK [vexxhost.ceph.cephadm : Log in to the container registry] **************** 2026-08-18 14:43:08.415425 | node1 | Tuesday 18 August 2026 14:43:08 +0000 (0:00:00.611) 0:00:43.237 ******** 2026-08-18 14:43:08.490449 | node1 | skipping: [node1] 2026-08-18 14:43:08.490661 | node1 | skipping: [node2] 2026-08-18 14:43:08.490905 | node1 | skipping: [node3] 2026-08-18 14:43:08.491153 | node1 | 2026-08-18 14:43:08.491395 | node1 | TASK [vexxhost.ceph.cephadm : Pre-pull the Ceph image] ************************* 2026-08-18 14:43:08.491640 | node1 | Tuesday 18 August 2026 14:43:08 +0000 (0:00:00.076) 0:00:43.313 ******** 2026-08-18 14:43:48.595208 | node1 | changed: [node3] 2026-08-18 14:43:48.595282 | node1 | changed: [node2] 2026-08-18 14:43:48.595296 | node1 | changed: [node1] 2026-08-18 14:43:48.595308 | node1 | 2026-08-18 14:43:48.595326 | node1 | TASK [vexxhost.ceph.cephadm : Pin CEPHADM_IMAGE for interactive login shells] *** 2026-08-18 14:43:48.595571 | node1 | Tuesday 18 August 2026 14:43:48 +0000 (0:00:40.103) 0:01:23.416 ******** 2026-08-18 14:43:49.212163 | node1 | changed: [node1] 2026-08-18 14:43:49.212210 | node1 | changed: [node2] 2026-08-18 14:43:49.212223 | node1 | changed: [node3] 2026-08-18 14:43:49.212236 | node1 | 2026-08-18 14:43:49.212248 | node1 | TASK [vexxhost.ceph.mon : Find this monitor's addresses on the public network] *** 2026-08-18 14:43:49.212260 | node1 | Tuesday 18 August 2026 14:43:49 +0000 (0:00:00.616) 0:01:24.033 ******** 2026-08-18 14:43:49.339159 | node1 | ok: [node1] 2026-08-18 14:43:49.339216 | node1 | ok: [node3] 2026-08-18 14:43:49.340155 | node1 | ok: [node2] 2026-08-18 14:43:49.340207 | node1 | 2026-08-18 14:43:49.340222 | node1 | TASK [vexxhost.ceph.mon : Assert exactly one address matches the public network] *** 2026-08-18 14:43:49.340234 | node1 | Tuesday 18 August 2026 14:43:49 +0000 (0:00:00.127) 0:01:24.160 ******** 2026-08-18 14:43:49.425096 | node1 | ok: [node1] => { 2026-08-18 14:43:49.425142 | node1 | "changed": false, 2026-08-18 14:43:49.425156 | node1 | "msg": "All assertions passed" 2026-08-18 14:43:49.425168 | node1 | } 2026-08-18 14:43:49.425180 | node1 | ok: [node2] => { 2026-08-18 14:43:49.425191 | node1 | "changed": false, 2026-08-18 14:43:49.425203 | node1 | "msg": "All assertions passed" 2026-08-18 14:43:49.425214 | node1 | } 2026-08-18 14:43:49.425225 | node1 | ok: [node3] => { 2026-08-18 14:43:49.425470 | node1 | "changed": false, 2026-08-18 14:43:49.425983 | node1 | "msg": "All assertions passed" 2026-08-18 14:43:49.426333 | node1 | } 2026-08-18 14:43:49.426648 | node1 | 2026-08-18 14:43:49.427064 | node1 | TASK [vexxhost.ceph.mon : Set this monitor's Ceph public IP] ******************* 2026-08-18 14:43:49.427402 | node1 | Tuesday 18 August 2026 14:43:49 +0000 (0:00:00.084) 0:01:24.245 ******** 2026-08-18 14:43:49.500468 | node1 | ok: [node1] 2026-08-18 14:43:49.500549 | node1 | ok: [node2] 2026-08-18 14:43:49.500658 | node1 | ok: [node3] 2026-08-18 14:43:49.500760 | node1 | 2026-08-18 14:43:49.500877 | node1 | TASK [vexxhost.ceph.mon : Check whether any monitor is already bootstrapped] *** 2026-08-18 14:43:49.500983 | node1 | Tuesday 18 August 2026 14:43:49 +0000 (0:00:00.077) 0:01:24.322 ******** 2026-08-18 14:43:50.171465 | node1 | ok: [node1] => (item=node1) 2026-08-18 14:43:50.171526 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 14:43:50.171651 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 14:43:50.171751 | node1 | 2026-08-18 14:43:50.171872 | node1 | TASK [vexxhost.ceph.mon : Select the bootstrap node (existing one, else the first monitor)] *** 2026-08-18 14:43:50.172023 | node1 | Tuesday 18 August 2026 14:43:50 +0000 (0:00:00.671) 0:01:24.993 ******** 2026-08-18 14:43:50.265442 | node1 | ok: [node1] 2026-08-18 14:43:50.265543 | node1 | ok: [node2] 2026-08-18 14:43:50.265650 | node1 | ok: [node3] 2026-08-18 14:43:50.265779 | node1 | 2026-08-18 14:43:50.265876 | node1 | TASK [vexxhost.ceph.mon : Bootstrap the cluster] ******************************* 2026-08-18 14:43:50.265995 | node1 | Tuesday 18 August 2026 14:43:50 +0000 (0:00:00.094) 0:01:25.088 ******** 2026-08-18 14:43:50.413688 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/mon/tasks/bootstrap.yml for node1, node2, node3 2026-08-18 14:43:50.413769 | node1 | 2026-08-18 14:43:50.413784 | node1 | TASK [vexxhost.ceph.mon : Ensure /etc/ceph exists] ***************************** 2026-08-18 14:43:50.413796 | node1 | Tuesday 18 August 2026 14:43:50 +0000 (0:00:00.145) 0:01:25.233 ******** 2026-08-18 14:43:50.675923 | node1 | skipping: [node2] 2026-08-18 14:43:50.675977 | node1 | skipping: [node3] 2026-08-18 14:43:50.675989 | node1 | changed: [node1] 2026-08-18 14:43:50.676002 | node1 | 2026-08-18 14:43:50.676051 | node1 | TASK [vexxhost.ceph.mon : Write the caller-provided SSH private key] *********** 2026-08-18 14:43:50.676082 | node1 | Tuesday 18 August 2026 14:43:50 +0000 (0:00:00.263) 0:01:25.496 ******** 2026-08-18 14:43:50.751082 | node1 | skipping: [node1] 2026-08-18 14:43:50.751141 | node1 | skipping: [node2] 2026-08-18 14:43:50.751151 | node1 | skipping: [node3] 2026-08-18 14:43:50.751160 | node1 | 2026-08-18 14:43:50.751170 | node1 | TASK [vexxhost.ceph.mon : Write the caller-provided SSH public key] ************ 2026-08-18 14:43:50.751179 | node1 | Tuesday 18 August 2026 14:43:50 +0000 (0:00:00.074) 0:01:25.571 ******** 2026-08-18 14:43:50.866099 | node1 | skipping: [node1] 2026-08-18 14:43:50.866156 | node1 | skipping: [node2] 2026-08-18 14:43:50.866453 | node1 | skipping: [node3] 2026-08-18 14:43:50.866740 | node1 | 2026-08-18 14:43:50.867107 | node1 | TASK [vexxhost.ceph.mon : Generate a cephadm SSH keypair when none was provided] *** 2026-08-18 14:43:50.867443 | node1 | Tuesday 18 August 2026 14:43:50 +0000 (0:00:00.115) 0:01:25.687 ******** 2026-08-18 14:43:51.930807 | node1 | skipping: [node2] 2026-08-18 14:43:51.931541 | node1 | skipping: [node3] 2026-08-18 14:43:51.931554 | node1 | changed: [node1] 2026-08-18 14:43:51.931559 | node1 | 2026-08-18 14:43:51.931564 | node1 | TASK [vexxhost.ceph.mon : Create a temporary ceph.conf for bootstrap overrides] *** 2026-08-18 14:43:51.931569 | node1 | Tuesday 18 August 2026 14:43:51 +0000 (0:00:01.066) 0:01:26.753 ******** 2026-08-18 14:43:52.255152 | node1 | skipping: [node2] 2026-08-18 14:43:52.255204 | node1 | skipping: [node3] 2026-08-18 14:43:52.255215 | node1 | changed: [node1] 2026-08-18 14:43:52.255224 | node1 | 2026-08-18 14:43:52.255239 | node1 | TASK [vexxhost.ceph.mon : Populate bootstrap ceph.conf overrides] ************** 2026-08-18 14:43:52.255417 | node1 | Tuesday 18 August 2026 14:43:52 +0000 (0:00:00.323) 0:01:27.076 ******** 2026-08-18 14:43:52.326369 | node1 | skipping: [node1] 2026-08-18 14:43:52.326575 | node1 | skipping: [node2] 2026-08-18 14:43:52.326820 | node1 | skipping: [node3] 2026-08-18 14:43:52.327077 | node1 | 2026-08-18 14:43:52.327337 | node1 | TASK [vexxhost.ceph.mon : Bootstrap the cluster] ******************************* 2026-08-18 14:43:52.327605 | node1 | Tuesday 18 August 2026 14:43:52 +0000 (0:00:00.072) 0:01:27.149 ******** 2026-08-18 14:44:35.707203 | node1 | skipping: [node2] 2026-08-18 14:44:35.707274 | node1 | skipping: [node3] 2026-08-18 14:44:35.707287 | node1 | changed: [node1] 2026-08-18 14:44:35.707305 | node1 | 2026-08-18 14:44:35.707723 | node1 | TASK [vexxhost.ceph.mon : Remove the temporary ceph.conf] ********************** 2026-08-18 14:44:35.707771 | node1 | Tuesday 18 August 2026 14:44:35 +0000 (0:00:43.379) 0:02:10.528 ******** 2026-08-18 14:44:35.976084 | node1 | skipping: [node2] 2026-08-18 14:44:35.976271 | node1 | skipping: [node3] 2026-08-18 14:44:35.976584 | node1 | changed: [node1] 2026-08-18 14:44:35.976870 | node1 | 2026-08-18 14:44:35.977216 | node1 | TASK [Register this monitor host] ********************************************** 2026-08-18 14:44:35.977562 | node1 | Tuesday 18 August 2026 14:44:35 +0000 (0:00:00.269) 0:02:10.798 ******** 2026-08-18 14:44:36.127074 | node1 | included: vexxhost.ceph.cephadm_host for node1, node2, node3 2026-08-18 14:44:36.127108 | node1 | 2026-08-18 14:44:36.127426 | node1 | TASK [vexxhost.ceph.cephadm_host : Get the cephadm public key] ***************** 2026-08-18 14:44:36.127767 | node1 | Tuesday 18 August 2026 14:44:36 +0000 (0:00:00.151) 0:02:10.949 ******** 2026-08-18 14:44:37.911543 | node1 | ok: [node1] 2026-08-18 14:44:37.912660 | node1 | 2026-08-18 14:44:37.912693 | node1 | TASK [vexxhost.ceph.cephadm_host : Resolve the cluster SSH public key to authorise] *** 2026-08-18 14:44:37.912707 | node1 | Tuesday 18 August 2026 14:44:37 +0000 (0:00:01.783) 0:02:12.733 ******** 2026-08-18 14:44:37.997098 | node1 | ok: [node1] 2026-08-18 14:44:37.997156 | node1 | ok: [node2] 2026-08-18 14:44:37.997164 | node1 | ok: [node3] 2026-08-18 14:44:37.997177 | node1 | 2026-08-18 14:44:37.997365 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert a cluster SSH key was resolved] ****** 2026-08-18 14:44:37.997521 | node1 | Tuesday 18 August 2026 14:44:37 +0000 (0:00:00.085) 0:02:12.819 ******** 2026-08-18 14:44:38.088647 | node1 | ok: [node1] => { 2026-08-18 14:44:38.089816 | node1 | "changed": false, 2026-08-18 14:44:38.089832 | node1 | "msg": "All assertions passed" 2026-08-18 14:44:38.089838 | node1 | } 2026-08-18 14:44:38.089843 | node1 | ok: [node2] => { 2026-08-18 14:44:38.089861 | node1 | "changed": false, 2026-08-18 14:44:38.089865 | node1 | "msg": "All assertions passed" 2026-08-18 14:44:38.089868 | node1 | } 2026-08-18 14:44:38.089872 | node1 | ok: [node3] => { 2026-08-18 14:44:38.089876 | node1 | "changed": false, 2026-08-18 14:44:38.089880 | node1 | "msg": "All assertions passed" 2026-08-18 14:44:38.089886 | node1 | } 2026-08-18 14:44:38.089984 | node1 | 2026-08-18 14:44:38.090086 | node1 | TASK [vexxhost.ceph.cephadm_host : Authorise the cluster SSH key for the cephadm user] *** 2026-08-18 14:44:38.090204 | node1 | Tuesday 18 August 2026 14:44:38 +0000 (0:00:00.091) 0:02:12.910 ******** 2026-08-18 14:44:38.534456 | node1 | changed: [node2] 2026-08-18 14:44:38.534558 | node1 | changed: [node3] 2026-08-18 14:44:38.534659 | node1 | ok: [node1] 2026-08-18 14:44:38.534765 | node1 | 2026-08-18 14:44:38.534887 | node1 | TASK [vexxhost.ceph.cephadm_host : Derive this host's cephadm labels from inventory membership] *** 2026-08-18 14:44:38.535035 | node1 | Tuesday 18 August 2026 14:44:38 +0000 (0:00:00.446) 0:02:13.357 ******** 2026-08-18 14:44:38.618822 | node1 | ok: [node1] 2026-08-18 14:44:38.619103 | node1 | ok: [node2] 2026-08-18 14:44:38.619411 | node1 | ok: [node3] 2026-08-18 14:44:38.619637 | node1 | 2026-08-18 14:44:38.619912 | node1 | TASK [vexxhost.ceph.cephadm_host : Find this host's addresses on the public network] *** 2026-08-18 14:44:38.620243 | node1 | Tuesday 18 August 2026 14:44:38 +0000 (0:00:00.083) 0:02:13.440 ******** 2026-08-18 14:44:38.738096 | node1 | ok: [node1] 2026-08-18 14:44:38.738323 | node1 | ok: [node2] 2026-08-18 14:44:38.738578 | node1 | ok: [node3] 2026-08-18 14:44:38.738767 | node1 | 2026-08-18 14:44:38.739120 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert exactly one address matches the public network] *** 2026-08-18 14:44:38.739391 | node1 | Tuesday 18 August 2026 14:44:38 +0000 (0:00:00.119) 0:02:13.560 ******** 2026-08-18 14:44:38.823079 | node1 | ok: [node1] => { 2026-08-18 14:44:38.823344 | node1 | "changed": false, 2026-08-18 14:44:38.823602 | node1 | "msg": "All assertions passed" 2026-08-18 14:44:38.823882 | node1 | } 2026-08-18 14:44:38.824179 | node1 | ok: [node2] => { 2026-08-18 14:44:38.824431 | node1 | "changed": false, 2026-08-18 14:44:38.824682 | node1 | "msg": "All assertions passed" 2026-08-18 14:44:38.824917 | node1 | } 2026-08-18 14:44:38.825211 | node1 | ok: [node3] => { 2026-08-18 14:44:38.825465 | node1 | "changed": false, 2026-08-18 14:44:38.825695 | node1 | "msg": "All assertions passed" 2026-08-18 14:44:38.825927 | node1 | } 2026-08-18 14:44:38.826184 | node1 | 2026-08-18 14:44:38.826519 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host (labels)] ***************** 2026-08-18 14:44:38.826938 | node1 | Tuesday 18 August 2026 14:44:38 +0000 (0:00:00.085) 0:02:13.645 ******** 2026-08-18 14:44:59.913227 | node1 | changed: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:44:59.913306 | node1 | changed: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:44:59.913563 | node1 | changed: [node1] 2026-08-18 14:44:59.913813 | node1 | 2026-08-18 14:44:59.914171 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host with a CRUSH location] **** 2026-08-18 14:44:59.914456 | node1 | Tuesday 18 August 2026 14:44:59 +0000 (0:00:21.089) 0:02:34.735 ******** 2026-08-18 14:45:00.023379 | node1 | skipping: [node1] 2026-08-18 14:45:00.023577 | node1 | skipping: [node2] 2026-08-18 14:45:00.023838 | node1 | skipping: [node3] 2026-08-18 14:45:00.024086 | node1 | 2026-08-18 14:45:00.024330 | node1 | TASK [vexxhost.ceph.mon : Apply the monitor service spec] ********************** 2026-08-18 14:45:00.024603 | node1 | Tuesday 18 August 2026 14:45:00 +0000 (0:00:00.110) 0:02:34.846 ******** 2026-08-18 14:45:03.298048 | node1 | changed: [node1] 2026-08-18 14:45:03.298093 | node1 | 2026-08-18 14:45:03.298100 | node1 | TASK [vexxhost.ceph.mon : Wait for this monitor's daemon to be present] ******** 2026-08-18 14:45:03.298195 | node1 | Tuesday 18 August 2026 14:45:03 +0000 (0:00:03.273) 0:02:38.120 ******** 2026-08-18 14:45:09.637670 | node1 | ok: [node1] 2026-08-18 14:45:09.638241 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:45:09.638660 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:45:09.639274 | node1 | 2026-08-18 14:45:09.639707 | node1 | TASK [vexxhost.ceph.mgr : Locate the cluster admin host] *********************** 2026-08-18 14:45:09.640195 | node1 | Tuesday 18 August 2026 14:45:09 +0000 (0:00:06.340) 0:02:44.460 ******** 2026-08-18 14:45:10.377482 | node1 | ok: [node1] => (item=node1) 2026-08-18 14:45:10.377546 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 14:45:10.377612 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 14:45:10.377954 | node1 | 2026-08-18 14:45:10.377975 | node1 | TASK [vexxhost.ceph.mgr : Select the admin host] ******************************* 2026-08-18 14:45:10.377981 | node1 | Tuesday 18 August 2026 14:45:10 +0000 (0:00:00.739) 0:02:45.199 ******** 2026-08-18 14:45:10.469400 | node1 | ok: [node1] 2026-08-18 14:45:10.469443 | node1 | ok: [node2] 2026-08-18 14:45:10.469450 | node1 | ok: [node3] 2026-08-18 14:45:10.469456 | node1 | 2026-08-18 14:45:10.469462 | node1 | TASK [Register this manager host] ********************************************** 2026-08-18 14:45:10.469467 | node1 | Tuesday 18 August 2026 14:45:10 +0000 (0:00:00.091) 0:02:45.291 ******** 2026-08-18 14:45:10.599627 | node1 | included: vexxhost.ceph.cephadm_host for node1, node2, node3 2026-08-18 14:45:10.599684 | node1 | 2026-08-18 14:45:10.599695 | node1 | TASK [vexxhost.ceph.cephadm_host : Get the cephadm public key] ***************** 2026-08-18 14:45:10.599705 | node1 | Tuesday 18 August 2026 14:45:10 +0000 (0:00:00.130) 0:02:45.421 ******** 2026-08-18 14:45:12.272129 | node1 | ok: [node1] 2026-08-18 14:45:12.272879 | node1 | 2026-08-18 14:45:12.272945 | node1 | TASK [vexxhost.ceph.cephadm_host : Resolve the cluster SSH public key to authorise] *** 2026-08-18 14:45:12.272961 | node1 | Tuesday 18 August 2026 14:45:12 +0000 (0:00:01.672) 0:02:47.093 ******** 2026-08-18 14:45:12.361422 | node1 | ok: [node1] 2026-08-18 14:45:12.361773 | node1 | ok: [node2] 2026-08-18 14:45:12.362150 | node1 | ok: [node3] 2026-08-18 14:45:12.362460 | node1 | 2026-08-18 14:45:12.362769 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert a cluster SSH key was resolved] ****** 2026-08-18 14:45:12.363062 | node1 | Tuesday 18 August 2026 14:45:12 +0000 (0:00:00.088) 0:02:47.182 ******** 2026-08-18 14:45:12.444642 | node1 | ok: [node1] => { 2026-08-18 14:45:12.444911 | node1 | "changed": false, 2026-08-18 14:45:12.445224 | node1 | "msg": "All assertions passed" 2026-08-18 14:45:12.445475 | node1 | } 2026-08-18 14:45:12.445736 | node1 | ok: [node2] => { 2026-08-18 14:45:12.445994 | node1 | "changed": false, 2026-08-18 14:45:12.446322 | node1 | "msg": "All assertions passed" 2026-08-18 14:45:12.446547 | node1 | } 2026-08-18 14:45:12.446801 | node1 | ok: [node3] => { 2026-08-18 14:45:12.447138 | node1 | "changed": false, 2026-08-18 14:45:12.447541 | node1 | "msg": "All assertions passed" 2026-08-18 14:45:12.447845 | node1 | } 2026-08-18 14:45:12.448088 | node1 | 2026-08-18 14:45:12.448253 | node1 | TASK [vexxhost.ceph.cephadm_host : Authorise the cluster SSH key for the cephadm user] *** 2026-08-18 14:45:12.448408 | node1 | Tuesday 18 August 2026 14:45:12 +0000 (0:00:00.084) 0:02:47.266 ******** 2026-08-18 14:45:12.814358 | node1 | ok: [node1] 2026-08-18 14:45:12.814528 | node1 | ok: [node2] 2026-08-18 14:45:12.814770 | node1 | ok: [node3] 2026-08-18 14:45:12.815057 | node1 | 2026-08-18 14:45:12.815392 | node1 | TASK [vexxhost.ceph.cephadm_host : Derive this host's cephadm labels from inventory membership] *** 2026-08-18 14:45:12.815658 | node1 | Tuesday 18 August 2026 14:45:12 +0000 (0:00:00.370) 0:02:47.636 ******** 2026-08-18 14:45:12.909631 | node1 | ok: [node1] 2026-08-18 14:45:12.909844 | node1 | ok: [node2] 2026-08-18 14:45:12.910125 | node1 | ok: [node3] 2026-08-18 14:45:12.910339 | node1 | 2026-08-18 14:45:12.910600 | node1 | TASK [vexxhost.ceph.cephadm_host : Find this host's addresses on the public network] *** 2026-08-18 14:45:12.910882 | node1 | Tuesday 18 August 2026 14:45:12 +0000 (0:00:00.095) 0:02:47.732 ******** 2026-08-18 14:45:13.033720 | node1 | ok: [node1] 2026-08-18 14:45:13.033906 | node1 | ok: [node2] 2026-08-18 14:45:13.034253 | node1 | ok: [node3] 2026-08-18 14:45:13.034420 | node1 | 2026-08-18 14:45:13.034683 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert exactly one address matches the public network] *** 2026-08-18 14:45:13.034934 | node1 | Tuesday 18 August 2026 14:45:13 +0000 (0:00:00.123) 0:02:47.856 ******** 2026-08-18 14:45:13.116880 | node1 | ok: [node1] => { 2026-08-18 14:45:13.117122 | node1 | "changed": false, 2026-08-18 14:45:13.117366 | node1 | "msg": "All assertions passed" 2026-08-18 14:45:13.117596 | node1 | } 2026-08-18 14:45:13.117832 | node1 | ok: [node2] => { 2026-08-18 14:45:13.118105 | node1 | "changed": false, 2026-08-18 14:45:13.118348 | node1 | "msg": "All assertions passed" 2026-08-18 14:45:13.118570 | node1 | } 2026-08-18 14:45:13.118863 | node1 | ok: [node3] => { 2026-08-18 14:45:13.119193 | node1 | "changed": false, 2026-08-18 14:45:13.119374 | node1 | "msg": "All assertions passed" 2026-08-18 14:45:13.119597 | node1 | } 2026-08-18 14:45:13.119817 | node1 | 2026-08-18 14:45:13.120098 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host (labels)] ***************** 2026-08-18 14:45:13.120336 | node1 | Tuesday 18 August 2026 14:45:13 +0000 (0:00:00.083) 0:02:47.939 ******** 2026-08-18 14:45:14.944698 | node1 | ok: [node1] 2026-08-18 14:45:14.944880 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:45:14.945172 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:45:14.945581 | node1 | 2026-08-18 14:45:14.945601 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host with a CRUSH location] **** 2026-08-18 14:45:14.945797 | node1 | Tuesday 18 August 2026 14:45:14 +0000 (0:00:01.827) 0:02:49.766 ******** 2026-08-18 14:45:15.054410 | node1 | skipping: [node1] 2026-08-18 14:45:15.054601 | node1 | skipping: [node2] 2026-08-18 14:45:15.054842 | node1 | skipping: [node3] 2026-08-18 14:45:15.055120 | node1 | 2026-08-18 14:45:15.055387 | node1 | TASK [vexxhost.ceph.mgr : Apply the manager service spec] ********************** 2026-08-18 14:45:15.055635 | node1 | Tuesday 18 August 2026 14:45:15 +0000 (0:00:00.110) 0:02:49.876 ******** 2026-08-18 14:45:23.469561 | node1 | changed: [node1] 2026-08-18 14:45:23.469624 | node1 | 2026-08-18 14:45:23.469638 | node1 | TASK [vexxhost.ceph.mgr : Wait for this manager's daemon to be present] ******** 2026-08-18 14:45:23.469647 | node1 | Tuesday 18 August 2026 14:45:23 +0000 (0:00:08.413) 0:02:58.290 ******** 2026-08-18 14:45:25.324465 | node1 | ok: [node1] 2026-08-18 14:45:25.324550 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:45:25.326087 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:45:25.326143 | node1 | 2026-08-18 14:45:25.326157 | node1 | TASK [vexxhost.ceph.mgr : Configure ceph-mgr modules] ************************** 2026-08-18 14:45:25.326170 | node1 | Tuesday 18 August 2026 14:45:25 +0000 (0:00:01.856) 0:03:00.146 ******** 2026-08-18 14:45:35.593563 | node1 | changed: [node1] => (item={'name': 'diskprediction_local'}) 2026-08-18 14:45:35.593645 | node1 | changed: [node1] => (item={'name': 'dashboard'}) 2026-08-18 14:45:35.593728 | node1 | ok: [node1] => (item={'name': 'iostat'}) 2026-08-18 14:45:35.593838 | node1 | changed: [node1] => (item={'name': 'prometheus'}) 2026-08-18 14:45:35.593945 | node1 | changed: [node1] => (item={'name': 'rgw'}) 2026-08-18 14:45:35.594066 | node1 | 2026-08-18 14:45:35.594178 | node1 | PLAY [Deploy Ceph OSDs] ******************************************************** 2026-08-18 14:45:35.594268 | node1 | 2026-08-18 14:45:35.594375 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:45:35.594484 | node1 | Tuesday 18 August 2026 14:45:35 +0000 (0:00:10.267) 0:03:10.414 ******** 2026-08-18 14:45:36.662977 | node1 | ok: [node1] 2026-08-18 14:45:36.663078 | node1 | ok: [node3] 2026-08-18 14:45:36.663164 | node1 | ok: [node2] 2026-08-18 14:45:36.663266 | node1 | 2026-08-18 14:45:36.663395 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:45:36.663498 | node1 | Tuesday 18 August 2026 14:45:36 +0000 (0:00:01.071) 0:03:11.485 ******** 2026-08-18 14:45:36.959046 | node1 | ok: [node1] 2026-08-18 14:45:36.959081 | node1 | ok: [node2] 2026-08-18 14:45:36.959087 | node1 | ok: [node3] 2026-08-18 14:45:36.959092 | node1 | 2026-08-18 14:45:36.959098 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:45:36.959104 | node1 | Tuesday 18 August 2026 14:45:36 +0000 (0:00:00.294) 0:03:11.780 ******** 2026-08-18 14:45:37.044450 | node1 | skipping: [node1] 2026-08-18 14:45:37.044536 | node1 | skipping: [node2] 2026-08-18 14:45:37.044642 | node1 | skipping: [node3] 2026-08-18 14:45:37.044744 | node1 | 2026-08-18 14:45:37.044859 | node1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-08-18 14:45:37.044966 | node1 | Tuesday 18 August 2026 14:45:37 +0000 (0:00:00.084) 0:03:11.865 ******** 2026-08-18 14:45:37.340299 | node1 | ok: [node1] 2026-08-18 14:45:37.340355 | node1 | ok: [node2] 2026-08-18 14:45:37.340820 | node1 | ok: [node3] 2026-08-18 14:45:37.340844 | node1 | 2026-08-18 14:45:37.340849 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:45:37.340854 | node1 | Tuesday 18 August 2026 14:45:37 +0000 (0:00:00.297) 0:03:12.163 ******** 2026-08-18 14:45:37.441427 | node1 | ok: [node1] => { 2026-08-18 14:45:37.441547 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:45:37.441653 | node1 | } 2026-08-18 14:45:37.441766 | node1 | ok: [node2] => { 2026-08-18 14:45:37.441871 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:45:37.441974 | node1 | } 2026-08-18 14:45:37.442148 | node1 | ok: [node3] => { 2026-08-18 14:45:37.442251 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:45:37.442354 | node1 | } 2026-08-18 14:45:37.442452 | node1 | 2026-08-18 14:45:37.442563 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:45:37.442675 | node1 | Tuesday 18 August 2026 14:45:37 +0000 (0:00:00.098) 0:03:12.261 ******** 2026-08-18 14:45:37.849600 | node1 | ok: [node1] 2026-08-18 14:45:37.849630 | node1 | ok: [node2] 2026-08-18 14:45:37.849635 | node1 | ok: [node3] 2026-08-18 14:45:37.849639 | node1 | 2026-08-18 14:45:37.849644 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:45:37.849648 | node1 | Tuesday 18 August 2026 14:45:37 +0000 (0:00:00.410) 0:03:12.672 ******** 2026-08-18 14:45:37.884656 | node1 | skipping: [node1] 2026-08-18 14:45:37.953764 | node1 | skipping: [node2] 2026-08-18 14:45:37.953791 | node1 | skipping: [node3] 2026-08-18 14:45:37.953795 | node1 | 2026-08-18 14:45:37.953800 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:45:37.953805 | node1 | Tuesday 18 August 2026 14:45:37 +0000 (0:00:00.101) 0:03:12.773 ******** 2026-08-18 14:45:37.981092 | node1 | skipping: [node1] 2026-08-18 14:45:38.055175 | node1 | skipping: [node2] 2026-08-18 14:45:38.055378 | node1 | skipping: [node3] 2026-08-18 14:45:38.058081 | node1 | 2026-08-18 14:45:38.058150 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:45:38.058166 | node1 | Tuesday 18 August 2026 14:45:38 +0000 (0:00:00.104) 0:03:12.878 ******** 2026-08-18 14:45:38.350886 | node1 | ok: [node1] 2026-08-18 14:45:38.350991 | node1 | ok: [node2] 2026-08-18 14:45:38.351162 | node1 | ok: [node3] 2026-08-18 14:45:38.351296 | node1 | 2026-08-18 14:45:38.351431 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:45:38.351568 | node1 | Tuesday 18 August 2026 14:45:38 +0000 (0:00:00.295) 0:03:13.173 ******** 2026-08-18 14:45:39.374874 | node1 | ok: [node2] 2026-08-18 14:45:39.374946 | node1 | ok: [node1] 2026-08-18 14:45:39.375392 | node1 | ok: [node3] 2026-08-18 14:45:39.375421 | node1 | 2026-08-18 14:45:39.375426 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:45:39.375431 | node1 | Tuesday 18 August 2026 14:45:39 +0000 (0:00:01.023) 0:03:14.197 ******** 2026-08-18 14:45:39.486048 | node1 | ok: [node1] => { 2026-08-18 14:45:39.486082 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:45:39.486087 | node1 | } 2026-08-18 14:45:39.486091 | node1 | ok: [node2] => { 2026-08-18 14:45:39.486095 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:45:39.486098 | node1 | } 2026-08-18 14:45:39.486102 | node1 | ok: [node3] => { 2026-08-18 14:45:39.486106 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:45:39.486110 | node1 | } 2026-08-18 14:45:39.486114 | node1 | 2026-08-18 14:45:39.486118 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:45:39.486122 | node1 | Tuesday 18 August 2026 14:45:39 +0000 (0:00:00.105) 0:03:14.302 ******** 2026-08-18 14:45:39.929659 | node1 | ok: [node1] 2026-08-18 14:45:39.930039 | node1 | ok: [node2] 2026-08-18 14:45:39.930358 | node1 | ok: [node3] 2026-08-18 14:45:39.930671 | node1 | 2026-08-18 14:45:39.930982 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:45:39.931326 | node1 | Tuesday 18 August 2026 14:45:39 +0000 (0:00:00.448) 0:03:14.751 ******** 2026-08-18 14:45:42.070737 | node1 | ok: [node2] 2026-08-18 14:45:42.071186 | node1 | ok: [node1] 2026-08-18 14:45:42.071528 | node1 | ok: [node3] 2026-08-18 14:45:42.071841 | node1 | 2026-08-18 14:45:42.072220 | node1 | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-08-18 14:45:42.072565 | node1 | Tuesday 18 August 2026 14:45:42 +0000 (0:00:02.141) 0:03:16.892 ******** 2026-08-18 14:45:42.137521 | node1 | skipping: [node1] 2026-08-18 14:45:42.137851 | node1 | skipping: [node2] 2026-08-18 14:45:42.138193 | node1 | skipping: [node3] 2026-08-18 14:45:42.138485 | node1 | 2026-08-18 14:45:42.138802 | node1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-08-18 14:45:42.139173 | node1 | Tuesday 18 August 2026 14:45:42 +0000 (0:00:00.067) 0:03:16.959 ******** 2026-08-18 14:45:42.212506 | node1 | skipping: [node1] 2026-08-18 14:45:42.212579 | node1 | skipping: [node2] 2026-08-18 14:45:42.212691 | node1 | skipping: [node3] 2026-08-18 14:45:42.212791 | node1 | 2026-08-18 14:45:42.212904 | node1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-08-18 14:45:42.213058 | node1 | Tuesday 18 August 2026 14:45:42 +0000 (0:00:00.075) 0:03:17.035 ******** 2026-08-18 14:45:42.300874 | node1 | skipping: [node1] 2026-08-18 14:45:42.300906 | node1 | skipping: [node2] 2026-08-18 14:45:42.300919 | node1 | skipping: [node3] 2026-08-18 14:45:42.300931 | node1 | 2026-08-18 14:45:42.300943 | node1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-08-18 14:45:42.300955 | node1 | Tuesday 18 August 2026 14:45:42 +0000 (0:00:00.086) 0:03:17.121 ******** 2026-08-18 14:45:43.246150 | node1 | ok: [node2] 2026-08-18 14:45:43.247280 | node1 | ok: [node1] 2026-08-18 14:45:43.247333 | node1 | ok: [node3] 2026-08-18 14:45:43.247380 | node1 | 2026-08-18 14:45:43.247394 | node1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-08-18 14:45:43.247407 | node1 | Tuesday 18 August 2026 14:45:43 +0000 (0:00:00.945) 0:03:18.067 ******** 2026-08-18 14:45:43.831791 | node1 | ok: [node1] 2026-08-18 14:45:43.833182 | node1 | ok: [node3] 2026-08-18 14:45:43.833211 | node1 | ok: [node2] 2026-08-18 14:45:43.833224 | node1 | 2026-08-18 14:45:43.833236 | node1 | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-08-18 14:45:43.833249 | node1 | Tuesday 18 August 2026 14:45:43 +0000 (0:00:00.586) 0:03:18.653 ******** 2026-08-18 14:45:45.022973 | node1 | ok: [node1] => (item={'path': '/etc/containerd'}) 2026-08-18 14:45:45.023030 | node1 | ok: [node2] => (item={'path': '/etc/containerd'}) 2026-08-18 14:45:45.023037 | node1 | ok: [node3] => (item={'path': '/etc/containerd'}) 2026-08-18 14:45:45.023043 | node1 | ok: [node1] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:45:45.023048 | node1 | ok: [node2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:45:45.023054 | node1 | ok: [node3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:45:45.023059 | node1 | ok: [node1] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:45:45.023064 | node1 | ok: [node2] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:45:45.023069 | node1 | ok: [node3] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:45:45.023074 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:45:45.023079 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:45:45.023084 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:45:45.023089 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:45:45.023094 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:45:45.023111 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:45:45.023115 | node1 | 2026-08-18 14:45:45.023119 | node1 | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-08-18 14:45:45.023123 | node1 | Tuesday 18 August 2026 14:45:45 +0000 (0:00:01.190) 0:03:19.843 ******** 2026-08-18 14:45:45.783062 | node1 | ok: [node1] 2026-08-18 14:45:45.783100 | node1 | ok: [node2] 2026-08-18 14:45:45.783106 | node1 | ok: [node3] 2026-08-18 14:45:45.783112 | node1 | 2026-08-18 14:45:45.783118 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:45:45.783287 | node1 | Tuesday 18 August 2026 14:45:45 +0000 (0:00:00.740) 0:03:20.584 ******** 2026-08-18 14:45:45.783468 | node1 | 2026-08-18 14:45:45.783663 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:45:45.783859 | node1 | Tuesday 18 August 2026 14:45:45 +0000 (0:00:00.006) 0:03:20.591 ******** 2026-08-18 14:45:45.784103 | node1 | 2026-08-18 14:45:45.784316 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:45:45.784518 | node1 | Tuesday 18 August 2026 14:45:45 +0000 (0:00:00.006) 0:03:20.598 ******** 2026-08-18 14:45:45.784698 | node1 | 2026-08-18 14:45:45.784898 | node1 | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-08-18 14:45:45.785114 | node1 | Tuesday 18 August 2026 14:45:45 +0000 (0:00:00.006) 0:03:20.605 ******** 2026-08-18 14:45:46.265291 | node1 | ok: [node1] 2026-08-18 14:45:46.265678 | node1 | ok: [node3] 2026-08-18 14:45:46.265993 | node1 | ok: [node2] 2026-08-18 14:45:46.266352 | node1 | 2026-08-18 14:45:46.266684 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:45:46.267239 | node1 | Tuesday 18 August 2026 14:45:46 +0000 (0:00:00.482) 0:03:21.087 ******** 2026-08-18 14:45:46.580159 | node1 | ok: [node1] 2026-08-18 14:45:46.580219 | node1 | ok: [node2] 2026-08-18 14:45:46.580229 | node1 | ok: [node3] 2026-08-18 14:45:46.580239 | node1 | 2026-08-18 14:45:46.580248 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:45:46.580407 | node1 | Tuesday 18 August 2026 14:45:46 +0000 (0:00:00.314) 0:03:21.401 ******** 2026-08-18 14:45:46.676102 | node1 | ok: [node1] => { 2026-08-18 14:45:46.676387 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:45:46.676630 | node1 | } 2026-08-18 14:45:46.676875 | node1 | ok: [node2] => { 2026-08-18 14:45:46.677274 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:45:46.677511 | node1 | } 2026-08-18 14:45:46.677756 | node1 | ok: [node3] => { 2026-08-18 14:45:46.678045 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:45:46.678329 | node1 | } 2026-08-18 14:45:46.678606 | node1 | 2026-08-18 14:45:46.678881 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:45:46.679179 | node1 | Tuesday 18 August 2026 14:45:46 +0000 (0:00:00.096) 0:03:21.498 ******** 2026-08-18 14:45:47.132518 | node1 | ok: [node1] 2026-08-18 14:45:47.132616 | node1 | ok: [node2] 2026-08-18 14:45:47.132724 | node1 | ok: [node3] 2026-08-18 14:45:47.132826 | node1 | 2026-08-18 14:45:47.132941 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:45:47.133087 | node1 | Tuesday 18 August 2026 14:45:47 +0000 (0:00:00.456) 0:03:21.955 ******** 2026-08-18 14:45:50.431167 | node1 | ok: [node2] 2026-08-18 14:45:50.431236 | node1 | ok: [node3] 2026-08-18 14:45:50.431249 | node1 | ok: [node1] 2026-08-18 14:45:50.431262 | node1 | 2026-08-18 14:45:50.431274 | node1 | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-08-18 14:45:50.431293 | node1 | Tuesday 18 August 2026 14:45:50 +0000 (0:00:03.298) 0:03:25.253 ******** 2026-08-18 14:45:51.351223 | node1 | ok: [node1] 2026-08-18 14:45:51.351438 | node1 | ok: [node3] 2026-08-18 14:45:51.351676 | node1 | ok: [node2] 2026-08-18 14:45:51.351901 | node1 | 2026-08-18 14:45:51.352180 | node1 | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-08-18 14:45:51.352429 | node1 | Tuesday 18 August 2026 14:45:51 +0000 (0:00:00.920) 0:03:26.173 ******** 2026-08-18 14:45:51.688702 | node1 | ok: [node1] 2026-08-18 14:45:51.688896 | node1 | ok: [node2] 2026-08-18 14:45:51.689185 | node1 | ok: [node3] 2026-08-18 14:45:51.689438 | node1 | 2026-08-18 14:45:51.689703 | node1 | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-08-18 14:45:51.690037 | node1 | Tuesday 18 August 2026 14:45:51 +0000 (0:00:00.337) 0:03:26.511 ******** 2026-08-18 14:45:52.272698 | node1 | ok: [node1] 2026-08-18 14:45:52.272752 | node1 | ok: [node2] 2026-08-18 14:45:52.272851 | node1 | ok: [node3] 2026-08-18 14:45:52.272950 | node1 | 2026-08-18 14:45:52.273086 | node1 | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-08-18 14:45:52.273210 | node1 | Tuesday 18 August 2026 14:45:52 +0000 (0:00:00.584) 0:03:27.095 ******** 2026-08-18 14:45:52.986152 | node1 | ok: [node1] => (item={'path': '/etc/docker'}) 2026-08-18 14:45:52.986217 | node1 | ok: [node3] => (item={'path': '/etc/docker'}) 2026-08-18 14:45:52.988095 | node1 | ok: [node2] => (item={'path': '/etc/docker'}) 2026-08-18 14:45:52.988158 | node1 | ok: [node1] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:45:52.988169 | node1 | ok: [node3] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:45:52.988177 | node1 | ok: [node2] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:45:52.988186 | node1 | ok: [node1] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:45:52.988208 | node1 | ok: [node3] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:45:52.988344 | node1 | ok: [node2] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:45:52.988644 | node1 | 2026-08-18 14:45:52.988969 | node1 | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-08-18 14:45:52.989339 | node1 | Tuesday 18 August 2026 14:45:52 +0000 (0:00:00.713) 0:03:27.808 ******** 2026-08-18 14:45:53.549284 | node1 | ok: [node1] 2026-08-18 14:45:53.549677 | node1 | ok: [node3] 2026-08-18 14:45:53.550055 | node1 | ok: [node2] 2026-08-18 14:45:53.550388 | node1 | 2026-08-18 14:45:53.550767 | node1 | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-08-18 14:45:53.551138 | node1 | Tuesday 18 August 2026 14:45:53 +0000 (0:00:00.563) 0:03:28.371 ******** 2026-08-18 14:45:54.163532 | node1 | ok: [node1] 2026-08-18 14:45:54.164176 | node1 | ok: [node2] 2026-08-18 14:45:54.167229 | node1 | ok: [node3] 2026-08-18 14:45:54.167321 | node1 | 2026-08-18 14:45:54.167443 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:45:54.167556 | node1 | Tuesday 18 August 2026 14:45:54 +0000 (0:00:00.594) 0:03:28.966 ******** 2026-08-18 14:45:54.167659 | node1 | 2026-08-18 14:45:54.167762 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:45:54.167875 | node1 | Tuesday 18 August 2026 14:45:54 +0000 (0:00:00.006) 0:03:28.972 ******** 2026-08-18 14:45:54.167975 | node1 | 2026-08-18 14:45:54.169053 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:45:54.169071 | node1 | Tuesday 18 August 2026 14:45:54 +0000 (0:00:00.006) 0:03:28.979 ******** 2026-08-18 14:45:54.169076 | node1 | 2026-08-18 14:45:54.169080 | node1 | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-08-18 14:45:54.169084 | node1 | Tuesday 18 August 2026 14:45:54 +0000 (0:00:00.006) 0:03:28.986 ******** 2026-08-18 14:45:54.660386 | node1 | ok: [node1] 2026-08-18 14:45:54.660431 | node1 | ok: [node3] 2026-08-18 14:45:54.660578 | node1 | ok: [node2] 2026-08-18 14:45:54.660863 | node1 | 2026-08-18 14:45:54.661209 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:45:54.661525 | node1 | Tuesday 18 August 2026 14:45:54 +0000 (0:00:00.495) 0:03:29.482 ******** 2026-08-18 14:45:54.742346 | node1 | ok: [node1] => { 2026-08-18 14:45:54.742847 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:45:54.743108 | node1 | } 2026-08-18 14:45:54.744601 | node1 | ok: [node2] => { 2026-08-18 14:45:54.744673 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:45:54.744724 | node1 | } 2026-08-18 14:45:54.744731 | node1 | ok: [node3] => { 2026-08-18 14:45:54.744764 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:45:54.744771 | node1 | } 2026-08-18 14:45:54.744812 | node1 | 2026-08-18 14:45:54.744819 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:45:54.744876 | node1 | Tuesday 18 August 2026 14:45:54 +0000 (0:00:00.082) 0:03:29.565 ******** 2026-08-18 14:45:55.116649 | node1 | ok: [node1] 2026-08-18 14:45:55.116705 | node1 | ok: [node2] 2026-08-18 14:45:55.116718 | node1 | ok: [node3] 2026-08-18 14:45:55.116730 | node1 | 2026-08-18 14:45:55.116743 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:45:55.116755 | node1 | Tuesday 18 August 2026 14:45:55 +0000 (0:00:00.372) 0:03:29.938 ******** 2026-08-18 14:45:55.202922 | node1 | skipping: [node1] 2026-08-18 14:45:55.202985 | node1 | skipping: [node2] 2026-08-18 14:45:55.202999 | node1 | skipping: [node3] 2026-08-18 14:45:55.203045 | node1 | 2026-08-18 14:45:55.203062 | node1 | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-08-18 14:45:55.203074 | node1 | Tuesday 18 August 2026 14:45:55 +0000 (0:00:00.086) 0:03:30.024 ******** 2026-08-18 14:45:55.289147 | node1 | ok: [node1] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:45:55.289196 | node1 | ok: [node2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:45:55.289209 | node1 | ok: [node3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:45:55.289221 | node1 | 2026-08-18 14:45:55.289234 | node1 | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-08-18 14:45:55.289246 | node1 | Tuesday 18 August 2026 14:45:55 +0000 (0:00:00.085) 0:03:30.110 ******** 2026-08-18 14:45:56.452955 | node1 | ok: [node2] 2026-08-18 14:45:56.452994 | node1 | ok: [node3] 2026-08-18 14:45:56.453021 | node1 | ok: [node1] 2026-08-18 14:45:56.453027 | node1 | 2026-08-18 14:45:56.453033 | node1 | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-08-18 14:45:56.453038 | node1 | Tuesday 18 August 2026 14:45:56 +0000 (0:00:01.161) 0:03:31.271 ******** 2026-08-18 14:45:57.292106 | node1 | ok: [node1] => (item=systemd-timesyncd) 2026-08-18 14:45:57.292143 | node1 | ok: [node2] => (item=systemd-timesyncd) 2026-08-18 14:45:57.292159 | node1 | ok: [node3] => (item=systemd-timesyncd) 2026-08-18 14:45:57.292165 | node1 | ok: [node2] => (item=ssh) 2026-08-18 14:45:57.292170 | node1 | ok: [node3] => (item=ssh) 2026-08-18 14:45:57.292175 | node1 | ok: [node1] => (item=ssh) 2026-08-18 14:45:57.292181 | node1 | 2026-08-18 14:45:57.292186 | node1 | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-08-18 14:45:57.292192 | node1 | Tuesday 18 August 2026 14:45:57 +0000 (0:00:00.842) 0:03:32.114 ******** 2026-08-18 14:45:57.573173 | node1 | ok: [node1] 2026-08-18 14:45:57.573299 | node1 | ok: [node2] 2026-08-18 14:45:57.573317 | node1 | ok: [node3] 2026-08-18 14:45:57.573331 | node1 | 2026-08-18 14:45:57.573344 | node1 | TASK [vexxhost.ceph.cephadm : Ensure the cephadm SSH user exists] ************** 2026-08-18 14:45:57.573356 | node1 | Tuesday 18 August 2026 14:45:57 +0000 (0:00:00.280) 0:03:32.394 ******** 2026-08-18 14:45:57.913558 | node1 | ok: [node2] 2026-08-18 14:45:57.913609 | node1 | ok: [node1] 2026-08-18 14:45:57.913621 | node1 | ok: [node3] 2026-08-18 14:45:57.913633 | node1 | 2026-08-18 14:45:57.913646 | node1 | TASK [vexxhost.ceph.cephadm : Grant passwordless sudo to the cephadm SSH user] *** 2026-08-18 14:45:57.913658 | node1 | Tuesday 18 August 2026 14:45:57 +0000 (0:00:00.340) 0:03:32.735 ******** 2026-08-18 14:45:58.524797 | node1 | ok: [node2] 2026-08-18 14:45:58.524853 | node1 | ok: [node1] 2026-08-18 14:45:58.524867 | node1 | ok: [node3] 2026-08-18 14:45:58.524879 | node1 | 2026-08-18 14:45:58.524891 | node1 | TASK [vexxhost.ceph.cephadm : Log in to the container registry] **************** 2026-08-18 14:45:58.524904 | node1 | Tuesday 18 August 2026 14:45:58 +0000 (0:00:00.609) 0:03:33.344 ******** 2026-08-18 14:45:58.596094 | node1 | skipping: [node1] 2026-08-18 14:45:58.596194 | node1 | skipping: [node2] 2026-08-18 14:45:58.596210 | node1 | skipping: [node3] 2026-08-18 14:45:58.596960 | node1 | 2026-08-18 14:45:58.596998 | node1 | TASK [vexxhost.ceph.cephadm : Pre-pull the Ceph image] ************************* 2026-08-18 14:45:58.597041 | node1 | Tuesday 18 August 2026 14:45:58 +0000 (0:00:00.073) 0:03:33.418 ******** 2026-08-18 14:45:59.060529 | node1 | ok: [node1] 2026-08-18 14:45:59.060630 | node1 | ok: [node2] 2026-08-18 14:45:59.061248 | node1 | ok: [node3] 2026-08-18 14:45:59.061279 | node1 | 2026-08-18 14:45:59.061284 | node1 | TASK [vexxhost.ceph.cephadm : Pin CEPHADM_IMAGE for interactive login shells] *** 2026-08-18 14:45:59.061289 | node1 | Tuesday 18 August 2026 14:45:59 +0000 (0:00:00.464) 0:03:33.882 ******** 2026-08-18 14:45:59.658110 | node1 | ok: [node1] 2026-08-18 14:45:59.658163 | node1 | ok: [node3] 2026-08-18 14:45:59.658179 | node1 | ok: [node2] 2026-08-18 14:45:59.658970 | node1 | 2026-08-18 14:45:59.659040 | node1 | TASK [vexxhost.ceph.osd : Locate the cluster admin host] *********************** 2026-08-18 14:45:59.659062 | node1 | Tuesday 18 August 2026 14:45:59 +0000 (0:00:00.597) 0:03:34.480 ******** 2026-08-18 14:46:00.392108 | node1 | ok: [node1] => (item=node1) 2026-08-18 14:46:00.392189 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 14:46:00.393364 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 14:46:00.393424 | node1 | 2026-08-18 14:46:00.393436 | node1 | TASK [vexxhost.ceph.osd : Select the admin host and its mon config path] ******* 2026-08-18 14:46:00.393447 | node1 | Tuesday 18 August 2026 14:46:00 +0000 (0:00:00.732) 0:03:35.213 ******** 2026-08-18 14:46:00.476128 | node1 | ok: [node1] 2026-08-18 14:46:00.476183 | node1 | ok: [node2] 2026-08-18 14:46:00.476199 | node1 | ok: [node3] 2026-08-18 14:46:00.476865 | node1 | 2026-08-18 14:46:00.476901 | node1 | TASK [vexxhost.ceph.osd : Set the admin mon config path] *********************** 2026-08-18 14:46:00.476911 | node1 | Tuesday 18 August 2026 14:46:00 +0000 (0:00:00.084) 0:03:35.297 ******** 2026-08-18 14:46:00.560472 | node1 | ok: [node1] 2026-08-18 14:46:00.560534 | node1 | ok: [node2] 2026-08-18 14:46:00.560640 | node1 | ok: [node3] 2026-08-18 14:46:00.560979 | node1 | 2026-08-18 14:46:00.561022 | node1 | TASK [Register this OSD host] ************************************************** 2026-08-18 14:46:00.561038 | node1 | Tuesday 18 August 2026 14:46:00 +0000 (0:00:00.084) 0:03:35.382 ******** 2026-08-18 14:46:00.681045 | node1 | included: vexxhost.ceph.cephadm_host for node1, node2, node3 2026-08-18 14:46:00.681084 | node1 | 2026-08-18 14:46:00.681092 | node1 | TASK [vexxhost.ceph.cephadm_host : Get the cephadm public key] ***************** 2026-08-18 14:46:00.681101 | node1 | Tuesday 18 August 2026 14:46:00 +0000 (0:00:00.120) 0:03:35.502 ******** 2026-08-18 14:46:02.353282 | node1 | ok: [node1] 2026-08-18 14:46:02.353410 | node1 | 2026-08-18 14:46:02.353420 | node1 | TASK [vexxhost.ceph.cephadm_host : Resolve the cluster SSH public key to authorise] *** 2026-08-18 14:46:02.353447 | node1 | Tuesday 18 August 2026 14:46:02 +0000 (0:00:01.671) 0:03:37.173 ******** 2026-08-18 14:46:02.438139 | node1 | ok: [node1] 2026-08-18 14:46:02.438191 | node1 | ok: [node2] 2026-08-18 14:46:02.438203 | node1 | ok: [node3] 2026-08-18 14:46:02.438215 | node1 | 2026-08-18 14:46:02.438228 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert a cluster SSH key was resolved] ****** 2026-08-18 14:46:02.438475 | node1 | Tuesday 18 August 2026 14:46:02 +0000 (0:00:00.085) 0:03:37.259 ******** 2026-08-18 14:46:02.523058 | node1 | ok: [node1] => { 2026-08-18 14:46:02.523094 | node1 | "changed": false, 2026-08-18 14:46:02.523106 | node1 | "msg": "All assertions passed" 2026-08-18 14:46:02.523118 | node1 | } 2026-08-18 14:46:02.523130 | node1 | ok: [node2] => { 2026-08-18 14:46:02.523141 | node1 | "changed": false, 2026-08-18 14:46:02.523152 | node1 | "msg": "All assertions passed" 2026-08-18 14:46:02.523164 | node1 | } 2026-08-18 14:46:02.523175 | node1 | ok: [node3] => { 2026-08-18 14:46:02.523186 | node1 | "changed": false, 2026-08-18 14:46:02.523197 | node1 | "msg": "All assertions passed" 2026-08-18 14:46:02.523477 | node1 | } 2026-08-18 14:46:02.523767 | node1 | 2026-08-18 14:46:02.524142 | node1 | TASK [vexxhost.ceph.cephadm_host : Authorise the cluster SSH key for the cephadm user] *** 2026-08-18 14:46:02.524465 | node1 | Tuesday 18 August 2026 14:46:02 +0000 (0:00:00.083) 0:03:37.342 ******** 2026-08-18 14:46:02.877701 | node1 | ok: [node1] 2026-08-18 14:46:02.878157 | node1 | ok: [node2] 2026-08-18 14:46:02.878483 | node1 | ok: [node3] 2026-08-18 14:46:02.878777 | node1 | 2026-08-18 14:46:02.879168 | node1 | TASK [vexxhost.ceph.cephadm_host : Derive this host's cephadm labels from inventory membership] *** 2026-08-18 14:46:02.879510 | node1 | Tuesday 18 August 2026 14:46:02 +0000 (0:00:00.356) 0:03:37.699 ******** 2026-08-18 14:46:02.958675 | node1 | ok: [node1] 2026-08-18 14:46:02.958735 | node1 | ok: [node2] 2026-08-18 14:46:02.958851 | node1 | ok: [node3] 2026-08-18 14:46:02.958954 | node1 | 2026-08-18 14:46:02.959279 | node1 | TASK [vexxhost.ceph.cephadm_host : Find this host's addresses on the public network] *** 2026-08-18 14:46:02.959340 | node1 | Tuesday 18 August 2026 14:46:02 +0000 (0:00:00.081) 0:03:37.780 ******** 2026-08-18 14:46:03.084773 | node1 | ok: [node1] 2026-08-18 14:46:03.084847 | node1 | ok: [node2] 2026-08-18 14:46:03.085032 | node1 | ok: [node3] 2026-08-18 14:46:03.085175 | node1 | 2026-08-18 14:46:03.085372 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert exactly one address matches the public network] *** 2026-08-18 14:46:03.085539 | node1 | Tuesday 18 August 2026 14:46:03 +0000 (0:00:00.126) 0:03:37.906 ******** 2026-08-18 14:46:03.170078 | node1 | ok: [node1] => { 2026-08-18 14:46:03.170159 | node1 | "changed": false, 2026-08-18 14:46:03.170169 | node1 | "msg": "All assertions passed" 2026-08-18 14:46:03.170178 | node1 | } 2026-08-18 14:46:03.170187 | node1 | ok: [node2] => { 2026-08-18 14:46:03.170196 | node1 | "changed": false, 2026-08-18 14:46:03.170204 | node1 | "msg": "All assertions passed" 2026-08-18 14:46:03.170212 | node1 | } 2026-08-18 14:46:03.170220 | node1 | ok: [node3] => { 2026-08-18 14:46:03.170573 | node1 | "changed": false, 2026-08-18 14:46:03.170892 | node1 | "msg": "All assertions passed" 2026-08-18 14:46:03.171224 | node1 | } 2026-08-18 14:46:03.171517 | node1 | 2026-08-18 14:46:03.171838 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host (labels)] ***************** 2026-08-18 14:46:03.172180 | node1 | Tuesday 18 August 2026 14:46:03 +0000 (0:00:00.082) 0:03:37.989 ******** 2026-08-18 14:46:05.050445 | node1 | ok: [node1] 2026-08-18 14:46:05.050512 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:46:05.050622 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:46:05.050728 | node1 | 2026-08-18 14:46:05.051030 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host with a CRUSH location] **** 2026-08-18 14:46:05.051074 | node1 | Tuesday 18 August 2026 14:46:05 +0000 (0:00:01.883) 0:03:39.872 ******** 2026-08-18 14:46:05.160370 | node1 | skipping: [node1] 2026-08-18 14:46:05.160444 | node1 | skipping: [node2] 2026-08-18 14:46:05.160457 | node1 | skipping: [node3] 2026-08-18 14:46:05.160470 | node1 | 2026-08-18 14:46:05.160482 | node1 | TASK [vexxhost.ceph.osd : Apply OSD service specs (drive groups)] ************** 2026-08-18 14:46:05.160495 | node1 | Tuesday 18 August 2026 14:46:05 +0000 (0:00:00.107) 0:03:39.980 ******** 2026-08-18 14:46:05.195167 | node1 | skipping: [node1] 2026-08-18 14:46:05.195250 | node1 | 2026-08-18 14:46:05.195821 | node1 | TASK [vexxhost.ceph.osd : List existing ceph-volume OSDs on this host] ********* 2026-08-18 14:46:05.195882 | node1 | Tuesday 18 August 2026 14:46:05 +0000 (0:00:00.036) 0:03:40.016 ******** 2026-08-18 14:46:06.743719 | node1 | ok: [node2] 2026-08-18 14:46:06.743810 | node1 | ok: [node3] 2026-08-18 14:46:06.743895 | node1 | ok: [node1] 2026-08-18 14:46:06.743999 | node1 | 2026-08-18 14:46:06.744153 | node1 | TASK [vexxhost.ceph.osd : Deploy OSDs on the explicitly listed devices] ******** 2026-08-18 14:46:06.744272 | node1 | Tuesday 18 August 2026 14:46:06 +0000 (0:00:01.548) 0:03:41.565 ******** 2026-08-18 14:52:01.284088 | node1 | changed: [node1] => (item=/dev/ceph-node1-osd0/data) 2026-08-18 14:52:01.284156 | node1 | changed: [node2 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node2-osd0/data) 2026-08-18 14:52:01.284170 | node1 | changed: [node3 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node3-osd0/data) 2026-08-18 14:52:01.284181 | node1 | changed: [node1] => (item=/dev/ceph-node1-osd1/data) 2026-08-18 14:52:01.284193 | node1 | changed: [node2 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node2-osd1/data) 2026-08-18 14:52:01.284204 | node1 | changed: [node3 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node3-osd1/data) 2026-08-18 14:52:01.284239 | node1 | changed: [node1] => (item=/dev/ceph-node1-osd2/data) 2026-08-18 14:52:01.284248 | node1 | changed: [node2 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node2-osd2/data) 2026-08-18 14:52:01.284256 | node1 | changed: [node3 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node3-osd2/data) 2026-08-18 14:52:01.284525 | node1 | 2026-08-18 14:52:01.284856 | node1 | TASK [vexxhost.ceph.osd : Wait for explicitly requested OSD daemons to be running on this host] *** 2026-08-18 14:52:01.285202 | node1 | Tuesday 18 August 2026 14:52:01 +0000 (0:05:54.538) 0:09:36.103 ******** 2026-08-18 14:52:23.237534 | node1 | FAILED - RETRYING: [node3 -> node1]: Wait for explicitly requested OSD daemons to be running on this host (120 retries left). 2026-08-18 14:52:23.237779 | node1 | ok: [node1] 2026-08-18 14:52:23.238104 | node1 | FAILED - RETRYING: [node2 -> node1]: Wait for explicitly requested OSD daemons to be running on this host (120 retries left). 2026-08-18 14:52:23.238392 | node1 | FAILED - RETRYING: [node3 -> node1]: Wait for explicitly requested OSD daemons to be running on this host (119 retries left). 2026-08-18 14:52:23.238656 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:52:23.238934 | node1 | FAILED - RETRYING: [node3 -> node1]: Wait for explicitly requested OSD daemons to be running on this host (118 retries left). 2026-08-18 14:52:23.239251 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:52:23.239594 | node1 | 2026-08-18 14:52:23.239934 | node1 | TASK [vexxhost.ceph.osd : Wait for OSD drive-group services to deploy at least one daemon] *** 2026-08-18 14:52:23.240313 | node1 | Tuesday 18 August 2026 14:52:23 +0000 (0:00:21.956) 0:09:58.059 ******** 2026-08-18 14:52:23.355447 | node1 | skipping: [node1] 2026-08-18 14:52:23.355697 | node1 | 2026-08-18 14:52:23.355924 | node1 | PLAY [Apply Ceph configuration] ************************************************ 2026-08-18 14:52:23.356155 | node1 | 2026-08-18 14:52:23.356484 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:52:23.356759 | node1 | Tuesday 18 August 2026 14:52:23 +0000 (0:00:00.119) 0:09:58.178 ******** 2026-08-18 14:52:24.520983 | node1 | ok: [node1] 2026-08-18 14:52:24.521435 | node1 | ok: [node3] 2026-08-18 14:52:24.521756 | node1 | ok: [node2] 2026-08-18 14:52:24.522333 | node1 | 2026-08-18 14:52:24.522421 | node1 | TASK [vexxhost.ceph.configure : Locate the cluster admin host] ***************** 2026-08-18 14:52:24.522793 | node1 | Tuesday 18 August 2026 14:52:24 +0000 (0:00:01.164) 0:09:59.343 ******** 2026-08-18 14:52:25.238077 | node1 | ok: [node1] => (item=node1) 2026-08-18 14:52:25.238125 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 14:52:25.238134 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 14:52:25.238284 | node1 | 2026-08-18 14:52:25.238552 | node1 | TASK [vexxhost.ceph.configure : Select the admin host] ************************* 2026-08-18 14:52:25.238896 | node1 | Tuesday 18 August 2026 14:52:25 +0000 (0:00:00.713) 0:10:00.056 ******** 2026-08-18 14:52:25.304289 | node1 | ok: [node1] 2026-08-18 14:52:25.304587 | node1 | ok: [node2] 2026-08-18 14:52:25.304837 | node1 | ok: [node3] 2026-08-18 14:52:25.305115 | node1 | 2026-08-18 14:52:25.305390 | node1 | TASK [vexxhost.ceph.configure : Set the admin mon config path] ***************** 2026-08-18 14:52:25.305830 | node1 | Tuesday 18 August 2026 14:52:25 +0000 (0:00:00.070) 0:10:00.127 ******** 2026-08-18 14:52:25.369130 | node1 | ok: [node1] 2026-08-18 14:52:25.369372 | node1 | ok: [node2] 2026-08-18 14:52:25.369619 | node1 | ok: [node3] 2026-08-18 14:52:25.369845 | node1 | 2026-08-18 14:52:25.370143 | node1 | TASK [vexxhost.ceph.configure : Apply central Ceph configuration] ************** 2026-08-18 14:52:25.370468 | node1 | Tuesday 18 August 2026 14:52:25 +0000 (0:00:00.064) 0:10:00.191 ******** 2026-08-18 14:52:25.446936 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/config.yml for node1, node2, node3 2026-08-18 14:52:25.447087 | node1 | 2026-08-18 14:52:25.447249 | node1 | TASK [vexxhost.ceph.configure : Apply central Ceph configuration] ************** 2026-08-18 14:52:25.447407 | node1 | Tuesday 18 August 2026 14:52:25 +0000 (0:00:00.077) 0:10:00.269 ******** 2026-08-18 14:52:45.137803 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/standby_behaviour) 2026-08-18 14:52:45.138101 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/RGW_API_SSL_VERIFY) 2026-08-18 14:52:45.138368 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/FEATURE_TOGGLE_CEPHFS) 2026-08-18 14:52:45.138669 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/FEATURE_TOGGLE_ISCSI) 2026-08-18 14:52:45.138979 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/FEATURE_TOGGLE_MIRRORING) 2026-08-18 14:52:45.139307 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/FEATURE_TOGGLE_NFS) 2026-08-18 14:52:45.139543 | node1 | 2026-08-18 14:52:45.139793 | node1 | TASK [vexxhost.ceph.configure : Configure erasure-code profiles] *************** 2026-08-18 14:52:45.140080 | node1 | Tuesday 18 August 2026 14:52:45 +0000 (0:00:19.690) 0:10:19.959 ******** 2026-08-18 14:52:45.241041 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/ec_profiles.yml for node1, node2, node3 2026-08-18 14:52:45.241298 | node1 | 2026-08-18 14:52:45.241607 | node1 | TASK [vexxhost.ceph.configure : Configure erasure-code profiles] *************** 2026-08-18 14:52:45.241906 | node1 | Tuesday 18 August 2026 14:52:45 +0000 (0:00:00.102) 0:10:20.062 ******** 2026-08-18 14:52:49.241190 | node1 | changed: [node1] => (item=ec_hdd_s3) 2026-08-18 14:52:49.241269 | node1 | 2026-08-18 14:52:49.241283 | node1 | TASK [vexxhost.ceph.configure : Configure CRUSH rules] ************************* 2026-08-18 14:52:49.241296 | node1 | Tuesday 18 August 2026 14:52:49 +0000 (0:00:03.998) 0:10:24.061 ******** 2026-08-18 14:52:49.323210 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/crush_rules.yml for node1, node2, node3 2026-08-18 14:52:49.323270 | node1 | 2026-08-18 14:52:49.323369 | node1 | TASK [vexxhost.ceph.configure : Configure CRUSH rules] ************************* 2026-08-18 14:52:49.323620 | node1 | Tuesday 18 August 2026 14:52:49 +0000 (0:00:00.083) 0:10:24.145 ******** 2026-08-18 14:53:01.032365 | node1 | changed: [node1] => (item=rep_hdd) 2026-08-18 14:53:01.032614 | node1 | changed: [node1] => (item=rep_lab) 2026-08-18 14:53:01.032864 | node1 | changed: [node1] => (item=ec_hdd_s3) 2026-08-18 14:53:01.033120 | node1 | 2026-08-18 14:53:01.033392 | node1 | TASK [vexxhost.ceph.configure : Configure pools] ******************************* 2026-08-18 14:53:01.033735 | node1 | Tuesday 18 August 2026 14:53:01 +0000 (0:00:11.709) 0:10:35.854 ******** 2026-08-18 14:53:01.137064 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/pools.yml for node1, node2, node3 2026-08-18 14:53:01.137828 | node1 | 2026-08-18 14:53:01.137897 | node1 | TASK [vexxhost.ceph.configure : Configure pools] ******************************* 2026-08-18 14:53:01.137912 | node1 | Tuesday 18 August 2026 14:53:01 +0000 (0:00:00.103) 0:10:35.958 ******** 2026-08-18 14:53:36.949211 | node1 | changed: [node1] => (item=molecule-1.rgw.buckets.data) 2026-08-18 14:53:36.949291 | node1 | changed: [node1] => (item=molecule-1.rgw.buckets.index) 2026-08-18 14:53:36.949306 | node1 | changed: [node1] => (item=molecule-1.rgw.buckets.non-ec) 2026-08-18 14:53:36.949318 | node1 | changed: [node1] => (item=molecule-1.rgw.control) 2026-08-18 14:53:36.949338 | node1 | changed: [node1] => (item=molecule-1.rgw.log) 2026-08-18 14:53:36.949932 | node1 | changed: [node1] => (item=molecule-1.rgw.meta) 2026-08-18 14:53:36.950087 | node1 | 2026-08-18 14:53:36.950101 | node1 | TASK [vexxhost.ceph.configure : Configure CephX keys] ************************** 2026-08-18 14:53:36.950110 | node1 | Tuesday 18 August 2026 14:53:36 +0000 (0:00:35.802) 0:11:11.761 ******** 2026-08-18 14:53:36.994783 | node1 | skipping: [node1] 2026-08-18 14:53:36.994915 | node1 | skipping: [node2] 2026-08-18 14:53:36.995104 | node1 | skipping: [node3] 2026-08-18 14:53:36.995567 | node1 | 2026-08-18 14:53:36.995614 | node1 | TASK [vexxhost.ceph.configure : Enroll the dashboard TLS certificate] ********** 2026-08-18 14:53:36.995622 | node1 | Tuesday 18 August 2026 14:53:36 +0000 (0:00:00.055) 0:11:11.816 ******** 2026-08-18 14:53:37.102811 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/dashboard.yml for node1, node2, node3 2026-08-18 14:53:37.102887 | node1 | 2026-08-18 14:53:37.102898 | node1 | TASK [vexxhost.ceph.configure : Check whether the dashboard mgr module is enabled] *** 2026-08-18 14:53:37.102908 | node1 | Tuesday 18 August 2026 14:53:37 +0000 (0:00:00.106) 0:11:11.923 ******** 2026-08-18 14:53:38.902554 | node1 | ok: [node1] 2026-08-18 14:53:38.902667 | node1 | 2026-08-18 14:53:38.902716 | node1 | TASK [vexxhost.ceph.configure : Get the current dashboard certificate] ********* 2026-08-18 14:53:38.902844 | node1 | Tuesday 18 August 2026 14:53:38 +0000 (0:00:01.800) 0:11:13.723 ******** 2026-08-18 14:53:40.820386 | node1 | ok: [node1] 2026-08-18 14:53:40.820473 | node1 | 2026-08-18 14:53:40.820483 | node1 | TASK [vexxhost.ceph.configure : Set the dashboard certificate] ***************** 2026-08-18 14:53:40.820501 | node1 | Tuesday 18 August 2026 14:53:40 +0000 (0:00:01.917) 0:11:15.641 ******** 2026-08-18 14:53:40.865871 | node1 | skipping: [node1] 2026-08-18 14:53:40.865895 | node1 | 2026-08-18 14:53:40.865903 | node1 | TASK [vexxhost.ceph.configure : Set the dashboard certificate key] ************* 2026-08-18 14:53:40.865911 | node1 | Tuesday 18 August 2026 14:53:40 +0000 (0:00:00.046) 0:11:15.687 ******** 2026-08-18 14:53:40.908325 | node1 | skipping: [node1] 2026-08-18 14:53:40.908359 | node1 | 2026-08-18 14:53:40.908372 | node1 | TASK [vexxhost.ceph.configure : Flag the dashboard for reload] ***************** 2026-08-18 14:53:40.908384 | node1 | Tuesday 18 August 2026 14:53:40 +0000 (0:00:00.041) 0:11:15.729 ******** 2026-08-18 14:53:40.953287 | node1 | skipping: [node1] 2026-08-18 14:53:40.953321 | node1 | 2026-08-18 14:53:40.953333 | node1 | TASK [vexxhost.ceph.configure : Create a self-signed certificate] ************** 2026-08-18 14:53:40.953345 | node1 | Tuesday 18 August 2026 14:53:40 +0000 (0:00:00.045) 0:11:15.775 ******** 2026-08-18 14:53:43.032160 | node1 | changed: [node1] 2026-08-18 14:53:43.032234 | node1 | 2026-08-18 14:53:43.032706 | node1 | TASK [vexxhost.ceph.configure : Flag the dashboard for reload] ***************** 2026-08-18 14:53:43.032785 | node1 | Tuesday 18 August 2026 14:53:43 +0000 (0:00:02.078) 0:11:17.853 ******** 2026-08-18 14:53:43.097875 | node1 | ok: [node1] 2026-08-18 14:53:43.097913 | node1 | 2026-08-18 14:53:43.097927 | node1 | TASK [vexxhost.ceph.configure : Reload the dashboard so it picks up the certificate] *** 2026-08-18 14:53:43.097945 | node1 | Tuesday 18 August 2026 14:53:43 +0000 (0:00:00.065) 0:11:17.919 ******** 2026-08-18 14:53:50.364306 | node1 | changed: [node1] => (item=ceph mgr module disable dashboard) 2026-08-18 14:53:50.364425 | node1 | changed: [node1] => (item=ceph mgr module enable dashboard) 2026-08-18 14:53:50.364440 | node1 | changed: [node1] => (item=ceph orch reconfig mgr) 2026-08-18 14:53:50.364453 | node1 | 2026-08-18 14:53:50.364466 | node1 | TASK [vexxhost.ceph.configure : Configure dashboard users] ********************* 2026-08-18 14:53:50.364479 | node1 | Tuesday 18 August 2026 14:53:50 +0000 (0:00:07.262) 0:11:25.181 ******** 2026-08-18 14:53:50.454387 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/dashboard_users.yml for node1, node2, node3 2026-08-18 14:53:50.454477 | node1 | 2026-08-18 14:53:50.454595 | node1 | TASK [vexxhost.ceph.configure : Configure dashboard users] ********************* 2026-08-18 14:53:50.454710 | node1 | Tuesday 18 August 2026 14:53:50 +0000 (0:00:00.095) 0:11:25.277 ******** 2026-08-18 14:54:01.041995 | node1 | changed: [node1] => (item=None) 2026-08-18 14:54:01.042076 | node1 | changed: [node1] => (item=None) 2026-08-18 14:54:01.042085 | node1 | changed: [node1 -> {{ ceph_configure_admin_host }}] 2026-08-18 14:54:01.042092 | node1 | 2026-08-18 14:54:01.042100 | node1 | PLAY [Deploy Ceph RADOS Gateways] ********************************************** 2026-08-18 14:54:01.042108 | node1 | 2026-08-18 14:54:01.042115 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:54:01.042123 | node1 | Tuesday 18 August 2026 14:54:01 +0000 (0:00:10.586) 0:11:35.863 ******** 2026-08-18 14:54:02.277597 | node1 | ok: [node1] 2026-08-18 14:54:02.277660 | node1 | ok: [node3] 2026-08-18 14:54:02.277673 | node1 | ok: [node2] 2026-08-18 14:54:02.277707 | node1 | 2026-08-18 14:54:02.277718 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:54:02.277728 | node1 | Tuesday 18 August 2026 14:54:02 +0000 (0:00:01.234) 0:11:37.098 ******** 2026-08-18 14:54:02.580798 | node1 | ok: [node2] 2026-08-18 14:54:02.580840 | node1 | ok: [node1] 2026-08-18 14:54:02.580846 | node1 | ok: [node3] 2026-08-18 14:54:02.580851 | node1 | 2026-08-18 14:54:02.580857 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:54:02.580863 | node1 | Tuesday 18 August 2026 14:54:02 +0000 (0:00:00.304) 0:11:37.402 ******** 2026-08-18 14:54:02.687535 | node1 | skipping: [node1] 2026-08-18 14:54:02.687601 | node1 | skipping: [node2] 2026-08-18 14:54:02.687614 | node1 | skipping: [node3] 2026-08-18 14:54:02.687626 | node1 | 2026-08-18 14:54:02.687638 | node1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-08-18 14:54:02.687651 | node1 | Tuesday 18 August 2026 14:54:02 +0000 (0:00:00.105) 0:11:37.507 ******** 2026-08-18 14:54:02.994442 | node1 | ok: [node2] 2026-08-18 14:54:02.994505 | node1 | ok: [node1] 2026-08-18 14:54:02.994515 | node1 | ok: [node3] 2026-08-18 14:54:02.994522 | node1 | 2026-08-18 14:54:02.994530 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:54:02.994538 | node1 | Tuesday 18 August 2026 14:54:02 +0000 (0:00:00.308) 0:11:37.816 ******** 2026-08-18 14:54:03.098580 | node1 | ok: [node1] => { 2026-08-18 14:54:03.098715 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:54:03.098848 | node1 | } 2026-08-18 14:54:03.098965 | node1 | ok: [node2] => { 2026-08-18 14:54:03.099105 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:54:03.099206 | node1 | } 2026-08-18 14:54:03.099324 | node1 | ok: [node3] => { 2026-08-18 14:54:03.099433 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:54:03.099537 | node1 | } 2026-08-18 14:54:03.099638 | node1 | 2026-08-18 14:54:03.099753 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:54:03.099864 | node1 | Tuesday 18 August 2026 14:54:03 +0000 (0:00:00.101) 0:11:37.917 ******** 2026-08-18 14:54:03.529034 | node1 | ok: [node2] 2026-08-18 14:54:03.529136 | node1 | ok: [node3] 2026-08-18 14:54:03.529294 | node1 | ok: [node1] 2026-08-18 14:54:03.529436 | node1 | 2026-08-18 14:54:03.529598 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:54:03.529767 | node1 | Tuesday 18 August 2026 14:54:03 +0000 (0:00:00.433) 0:11:38.351 ******** 2026-08-18 14:54:03.735473 | node1 | skipping: [node1] 2026-08-18 14:54:03.735532 | node1 | skipping: [node2] 2026-08-18 14:54:03.735541 | node1 | skipping: [node3] 2026-08-18 14:54:03.735549 | node1 | 2026-08-18 14:54:03.735557 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:54:03.735565 | node1 | Tuesday 18 August 2026 14:54:03 +0000 (0:00:00.203) 0:11:38.554 ******** 2026-08-18 14:54:03.849633 | node1 | skipping: [node1] 2026-08-18 14:54:03.849726 | node1 | skipping: [node2] 2026-08-18 14:54:03.849845 | node1 | skipping: [node3] 2026-08-18 14:54:03.849951 | node1 | 2026-08-18 14:54:03.850084 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:54:03.850209 | node1 | Tuesday 18 August 2026 14:54:03 +0000 (0:00:00.116) 0:11:38.671 ******** 2026-08-18 14:54:04.161578 | node1 | ok: [node2] 2026-08-18 14:54:04.161672 | node1 | ok: [node1] 2026-08-18 14:54:04.161752 | node1 | ok: [node3] 2026-08-18 14:54:04.161861 | node1 | 2026-08-18 14:54:04.161978 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:54:04.162124 | node1 | Tuesday 18 August 2026 14:54:04 +0000 (0:00:00.309) 0:11:38.980 ******** 2026-08-18 14:54:05.252151 | node1 | ok: [node2] 2026-08-18 14:54:05.252247 | node1 | ok: [node3] 2026-08-18 14:54:05.252331 | node1 | ok: [node1] 2026-08-18 14:54:05.252434 | node1 | 2026-08-18 14:54:05.252552 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:54:05.252656 | node1 | Tuesday 18 August 2026 14:54:05 +0000 (0:00:01.094) 0:11:40.074 ******** 2026-08-18 14:54:05.386401 | node1 | ok: [node1] => { 2026-08-18 14:54:05.386471 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:54:05.386599 | node1 | } 2026-08-18 14:54:05.386741 | node1 | ok: [node2] => { 2026-08-18 14:54:05.386908 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:54:05.388158 | node1 | } 2026-08-18 14:54:05.388219 | node1 | ok: [node3] => { 2026-08-18 14:54:05.388224 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:54:05.388229 | node1 | } 2026-08-18 14:54:05.388233 | node1 | 2026-08-18 14:54:05.388237 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:54:05.388242 | node1 | Tuesday 18 August 2026 14:54:05 +0000 (0:00:00.134) 0:11:40.209 ******** 2026-08-18 14:54:05.860402 | node1 | ok: [node1] 2026-08-18 14:54:05.860463 | node1 | ok: [node2] 2026-08-18 14:54:05.860473 | node1 | ok: [node3] 2026-08-18 14:54:05.860483 | node1 | 2026-08-18 14:54:05.860492 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:54:05.860502 | node1 | Tuesday 18 August 2026 14:54:05 +0000 (0:00:00.472) 0:11:40.681 ******** 2026-08-18 14:54:08.128615 | node1 | ok: [node2] 2026-08-18 14:54:08.128691 | node1 | ok: [node3] 2026-08-18 14:54:08.128794 | node1 | ok: [node1] 2026-08-18 14:54:08.128946 | node1 | 2026-08-18 14:54:08.129119 | node1 | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-08-18 14:54:08.129218 | node1 | Tuesday 18 August 2026 14:54:08 +0000 (0:00:02.268) 0:11:42.950 ******** 2026-08-18 14:54:08.205625 | node1 | skipping: [node1] 2026-08-18 14:54:08.205689 | node1 | skipping: [node2] 2026-08-18 14:54:08.205807 | node1 | skipping: [node3] 2026-08-18 14:54:08.205908 | node1 | 2026-08-18 14:54:08.206039 | node1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-08-18 14:54:08.206165 | node1 | Tuesday 18 August 2026 14:54:08 +0000 (0:00:00.076) 0:11:43.027 ******** 2026-08-18 14:54:08.287593 | node1 | skipping: [node1] 2026-08-18 14:54:08.287734 | node1 | skipping: [node2] 2026-08-18 14:54:08.287802 | node1 | skipping: [node3] 2026-08-18 14:54:08.287917 | node1 | 2026-08-18 14:54:08.288063 | node1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-08-18 14:54:08.288195 | node1 | Tuesday 18 August 2026 14:54:08 +0000 (0:00:00.082) 0:11:43.109 ******** 2026-08-18 14:54:08.361327 | node1 | skipping: [node1] 2026-08-18 14:54:08.361393 | node1 | skipping: [node2] 2026-08-18 14:54:08.361406 | node1 | skipping: [node3] 2026-08-18 14:54:08.361418 | node1 | 2026-08-18 14:54:08.361430 | node1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-08-18 14:54:08.361442 | node1 | Tuesday 18 August 2026 14:54:08 +0000 (0:00:00.072) 0:11:43.181 ******** 2026-08-18 14:54:09.466356 | node1 | ok: [node2] 2026-08-18 14:54:09.466412 | node1 | ok: [node1] 2026-08-18 14:54:09.466419 | node1 | ok: [node3] 2026-08-18 14:54:09.466424 | node1 | 2026-08-18 14:54:09.466430 | node1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-08-18 14:54:09.466437 | node1 | Tuesday 18 August 2026 14:54:09 +0000 (0:00:01.105) 0:11:44.286 ******** 2026-08-18 14:54:10.088905 | node1 | ok: [node2] 2026-08-18 14:54:10.088974 | node1 | ok: [node3] 2026-08-18 14:54:10.089114 | node1 | ok: [node1] 2026-08-18 14:54:10.089221 | node1 | 2026-08-18 14:54:10.089354 | node1 | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-08-18 14:54:10.089488 | node1 | Tuesday 18 August 2026 14:54:10 +0000 (0:00:00.621) 0:11:44.908 ******** 2026-08-18 14:54:11.297082 | node1 | ok: [node2] => (item={'path': '/etc/containerd'}) 2026-08-18 14:54:11.298167 | node1 | ok: [node1] => (item={'path': '/etc/containerd'}) 2026-08-18 14:54:11.298299 | node1 | ok: [node3] => (item={'path': '/etc/containerd'}) 2026-08-18 14:54:11.298414 | node1 | ok: [node2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:54:11.298532 | node1 | ok: [node1] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:54:11.300027 | node1 | ok: [node3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:54:11.300084 | node1 | ok: [node2] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:54:11.300088 | node1 | ok: [node1] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:54:11.300092 | node1 | ok: [node3] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:54:11.300096 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:54:11.300101 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:54:11.300105 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:54:11.300108 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:54:11.300112 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:54:11.300116 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:54:11.300120 | node1 | 2026-08-18 14:54:11.300124 | node1 | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-08-18 14:54:11.300128 | node1 | Tuesday 18 August 2026 14:54:11 +0000 (0:00:01.211) 0:11:46.119 ******** 2026-08-18 14:54:12.058052 | node1 | ok: [node2] 2026-08-18 14:54:12.058097 | node1 | ok: [node3] 2026-08-18 14:54:12.058103 | node1 | ok: [node1] 2026-08-18 14:54:12.058109 | node1 | 2026-08-18 14:54:12.058115 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:54:12.058121 | node1 | Tuesday 18 August 2026 14:54:12 +0000 (0:00:00.713) 0:11:46.833 ******** 2026-08-18 14:54:12.058126 | node1 | 2026-08-18 14:54:12.058131 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:54:12.058136 | node1 | Tuesday 18 August 2026 14:54:12 +0000 (0:00:00.017) 0:11:46.850 ******** 2026-08-18 14:54:12.058141 | node1 | 2026-08-18 14:54:12.058309 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:54:12.058574 | node1 | Tuesday 18 August 2026 14:54:12 +0000 (0:00:00.013) 0:11:46.864 ******** 2026-08-18 14:54:12.058746 | node1 | 2026-08-18 14:54:12.058989 | node1 | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-08-18 14:54:12.059313 | node1 | Tuesday 18 August 2026 14:54:12 +0000 (0:00:00.013) 0:11:46.877 ******** 2026-08-18 14:54:12.544965 | node1 | ok: [node2] 2026-08-18 14:54:12.545240 | node1 | ok: [node1] 2026-08-18 14:54:12.545482 | node1 | ok: [node3] 2026-08-18 14:54:12.545717 | node1 | 2026-08-18 14:54:12.545952 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:54:12.546441 | node1 | Tuesday 18 August 2026 14:54:12 +0000 (0:00:00.488) 0:11:47.365 ******** 2026-08-18 14:54:12.844421 | node1 | ok: [node1] 2026-08-18 14:54:12.844614 | node1 | ok: [node2] 2026-08-18 14:54:12.844850 | node1 | ok: [node3] 2026-08-18 14:54:12.845608 | node1 | 2026-08-18 14:54:12.845622 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:54:12.845639 | node1 | Tuesday 18 August 2026 14:54:12 +0000 (0:00:00.300) 0:11:47.665 ******** 2026-08-18 14:54:12.957144 | node1 | ok: [node1] => { 2026-08-18 14:54:12.957393 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:54:12.957627 | node1 | } 2026-08-18 14:54:12.957863 | node1 | ok: [node2] => { 2026-08-18 14:54:12.958152 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:54:12.958382 | node1 | } 2026-08-18 14:54:12.958615 | node1 | ok: [node3] => { 2026-08-18 14:54:12.958870 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:54:12.959125 | node1 | } 2026-08-18 14:54:12.959452 | node1 | 2026-08-18 14:54:12.959712 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:54:12.959957 | node1 | Tuesday 18 August 2026 14:54:12 +0000 (0:00:00.113) 0:11:47.779 ******** 2026-08-18 14:54:13.429739 | node1 | ok: [node1] 2026-08-18 14:54:13.429944 | node1 | ok: [node2] 2026-08-18 14:54:13.430301 | node1 | ok: [node3] 2026-08-18 14:54:13.430547 | node1 | 2026-08-18 14:54:13.430839 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:54:13.431183 | node1 | Tuesday 18 August 2026 14:54:13 +0000 (0:00:00.471) 0:11:48.251 ******** 2026-08-18 14:54:16.796910 | node1 | ok: [node2] 2026-08-18 14:54:16.797143 | node1 | ok: [node3] 2026-08-18 14:54:16.797408 | node1 | ok: [node1] 2026-08-18 14:54:16.797633 | node1 | 2026-08-18 14:54:16.797885 | node1 | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-08-18 14:54:16.798175 | node1 | Tuesday 18 August 2026 14:54:16 +0000 (0:00:03.367) 0:11:51.619 ******** 2026-08-18 14:54:17.783609 | node1 | ok: [node2] 2026-08-18 14:54:17.783755 | node1 | ok: [node3] 2026-08-18 14:54:17.783800 | node1 | ok: [node1] 2026-08-18 14:54:17.783927 | node1 | 2026-08-18 14:54:17.784083 | node1 | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-08-18 14:54:17.784202 | node1 | Tuesday 18 August 2026 14:54:17 +0000 (0:00:00.985) 0:11:52.604 ******** 2026-08-18 14:54:18.078838 | node1 | ok: [node1] 2026-08-18 14:54:18.079110 | node1 | ok: [node2] 2026-08-18 14:54:18.079430 | node1 | ok: [node3] 2026-08-18 14:54:18.079655 | node1 | 2026-08-18 14:54:18.079936 | node1 | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-08-18 14:54:18.080282 | node1 | Tuesday 18 August 2026 14:54:18 +0000 (0:00:00.296) 0:11:52.900 ******** 2026-08-18 14:54:18.665360 | node1 | ok: [node2] 2026-08-18 14:54:18.665600 | node1 | ok: [node3] 2026-08-18 14:54:18.665891 | node1 | ok: [node1] 2026-08-18 14:54:18.666234 | node1 | 2026-08-18 14:54:18.666577 | node1 | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-08-18 14:54:18.666870 | node1 | Tuesday 18 August 2026 14:54:18 +0000 (0:00:00.586) 0:11:53.487 ******** 2026-08-18 14:54:19.394182 | node1 | ok: [node2] => (item={'path': '/etc/docker'}) 2026-08-18 14:54:19.394273 | node1 | ok: [node1] => (item={'path': '/etc/docker'}) 2026-08-18 14:54:19.394287 | node1 | ok: [node3] => (item={'path': '/etc/docker'}) 2026-08-18 14:54:19.394305 | node1 | ok: [node1] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:54:19.396094 | node1 | ok: [node2] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:54:19.396170 | node1 | ok: [node3] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:54:19.396186 | node1 | ok: [node2] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:54:19.396199 | node1 | ok: [node1] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:54:19.396210 | node1 | ok: [node3] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:54:19.396223 | node1 | 2026-08-18 14:54:19.396457 | node1 | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-08-18 14:54:19.396808 | node1 | Tuesday 18 August 2026 14:54:19 +0000 (0:00:00.728) 0:11:54.215 ******** 2026-08-18 14:54:20.001038 | node1 | ok: [node2] 2026-08-18 14:54:20.001441 | node1 | ok: [node1] 2026-08-18 14:54:20.001755 | node1 | ok: [node3] 2026-08-18 14:54:20.002082 | node1 | 2026-08-18 14:54:20.002418 | node1 | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-08-18 14:54:20.002735 | node1 | Tuesday 18 August 2026 14:54:19 +0000 (0:00:00.607) 0:11:54.822 ******** 2026-08-18 14:54:20.621889 | node1 | ok: [node2] 2026-08-18 14:54:20.622336 | node1 | ok: [node1] 2026-08-18 14:54:20.622658 | node1 | ok: [node3] 2026-08-18 14:54:20.622946 | node1 | 2026-08-18 14:54:20.623312 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:54:20.623635 | node1 | Tuesday 18 August 2026 14:54:20 +0000 (0:00:00.579) 0:11:55.401 ******** 2026-08-18 14:54:20.623985 | node1 | 2026-08-18 14:54:20.624378 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:54:20.624697 | node1 | Tuesday 18 August 2026 14:54:20 +0000 (0:00:00.014) 0:11:55.416 ******** 2026-08-18 14:54:20.624984 | node1 | 2026-08-18 14:54:20.625354 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:54:20.625692 | node1 | Tuesday 18 August 2026 14:54:20 +0000 (0:00:00.014) 0:11:55.430 ******** 2026-08-18 14:54:20.626045 | node1 | 2026-08-18 14:54:20.626671 | node1 | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-08-18 14:54:20.626689 | node1 | Tuesday 18 August 2026 14:54:20 +0000 (0:00:00.013) 0:11:55.443 ******** 2026-08-18 14:54:21.159121 | node1 | ok: [node1] 2026-08-18 14:54:21.159231 | node1 | ok: [node2] 2026-08-18 14:54:21.159243 | node1 | ok: [node3] 2026-08-18 14:54:21.159340 | node1 | 2026-08-18 14:54:21.159593 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:54:21.159843 | node1 | Tuesday 18 August 2026 14:54:21 +0000 (0:00:00.536) 0:11:55.980 ******** 2026-08-18 14:54:21.230966 | node1 | ok: [node1] => { 2026-08-18 14:54:21.231263 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:54:21.231494 | node1 | } 2026-08-18 14:54:21.231735 | node1 | ok: [node2] => { 2026-08-18 14:54:21.231983 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:54:21.232267 | node1 | } 2026-08-18 14:54:21.232521 | node1 | ok: [node3] => { 2026-08-18 14:54:21.232788 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:54:21.233042 | node1 | } 2026-08-18 14:54:21.233276 | node1 | 2026-08-18 14:54:21.233531 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:54:21.233777 | node1 | Tuesday 18 August 2026 14:54:21 +0000 (0:00:00.072) 0:11:56.053 ******** 2026-08-18 14:54:21.626186 | node1 | ok: [node1] 2026-08-18 14:54:21.626259 | node1 | ok: [node2] 2026-08-18 14:54:21.626269 | node1 | ok: [node3] 2026-08-18 14:54:21.626278 | node1 | 2026-08-18 14:54:21.626310 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:54:21.626385 | node1 | Tuesday 18 August 2026 14:54:21 +0000 (0:00:00.393) 0:11:56.447 ******** 2026-08-18 14:54:21.733509 | node1 | skipping: [node1] 2026-08-18 14:54:21.794027 | node1 | skipping: [node2] 2026-08-18 14:54:21.794298 | node1 | skipping: [node3] 2026-08-18 14:54:21.794577 | node1 | 2026-08-18 14:54:21.794859 | node1 | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-08-18 14:54:21.795171 | node1 | Tuesday 18 August 2026 14:54:21 +0000 (0:00:00.169) 0:11:56.616 ******** 2026-08-18 14:54:21.888285 | node1 | ok: [node1] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:54:21.888519 | node1 | ok: [node2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:54:21.888776 | node1 | ok: [node3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:54:21.888998 | node1 | 2026-08-18 14:54:21.889284 | node1 | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-08-18 14:54:21.889530 | node1 | Tuesday 18 August 2026 14:54:21 +0000 (0:00:00.094) 0:11:56.710 ******** 2026-08-18 14:54:23.806098 | node1 | ok: [node3] 2026-08-18 14:54:23.806390 | node1 | ok: [node1] 2026-08-18 14:54:23.806860 | node1 | ok: [node2] 2026-08-18 14:54:23.807178 | node1 | 2026-08-18 14:54:23.807487 | node1 | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-08-18 14:54:23.807792 | node1 | Tuesday 18 August 2026 14:54:23 +0000 (0:00:01.917) 0:11:58.627 ******** 2026-08-18 14:54:24.690240 | node1 | ok: [node1] => (item=systemd-timesyncd) 2026-08-18 14:54:24.690444 | node1 | ok: [node2] => (item=systemd-timesyncd) 2026-08-18 14:54:24.690688 | node1 | ok: [node3] => (item=systemd-timesyncd) 2026-08-18 14:54:24.690929 | node1 | ok: [node1] => (item=ssh) 2026-08-18 14:54:24.691206 | node1 | ok: [node2] => (item=ssh) 2026-08-18 14:54:24.691636 | node1 | ok: [node3] => (item=ssh) 2026-08-18 14:54:24.691672 | node1 | 2026-08-18 14:54:24.691950 | node1 | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-08-18 14:54:24.692252 | node1 | Tuesday 18 August 2026 14:54:24 +0000 (0:00:00.884) 0:11:59.512 ******** 2026-08-18 14:54:24.981526 | node1 | ok: [node2] 2026-08-18 14:54:24.981756 | node1 | ok: [node1] 2026-08-18 14:54:24.981992 | node1 | ok: [node3] 2026-08-18 14:54:24.982251 | node1 | 2026-08-18 14:54:24.982501 | node1 | TASK [vexxhost.ceph.cephadm : Ensure the cephadm SSH user exists] ************** 2026-08-18 14:54:24.982751 | node1 | Tuesday 18 August 2026 14:54:24 +0000 (0:00:00.291) 0:11:59.803 ******** 2026-08-18 14:54:25.340465 | node1 | ok: [node2] 2026-08-18 14:54:25.340694 | node1 | ok: [node3] 2026-08-18 14:54:25.340935 | node1 | ok: [node1] 2026-08-18 14:54:25.341194 | node1 | 2026-08-18 14:54:25.341461 | node1 | TASK [vexxhost.ceph.cephadm : Grant passwordless sudo to the cephadm SSH user] *** 2026-08-18 14:54:25.341709 | node1 | Tuesday 18 August 2026 14:54:25 +0000 (0:00:00.359) 0:12:00.162 ******** 2026-08-18 14:54:25.915541 | node1 | ok: [node1] 2026-08-18 14:54:25.915588 | node1 | ok: [node2] 2026-08-18 14:54:25.915594 | node1 | ok: [node3] 2026-08-18 14:54:25.915599 | node1 | 2026-08-18 14:54:25.915605 | node1 | TASK [vexxhost.ceph.cephadm : Log in to the container registry] **************** 2026-08-18 14:54:25.915612 | node1 | Tuesday 18 August 2026 14:54:25 +0000 (0:00:00.573) 0:12:00.736 ******** 2026-08-18 14:54:25.990956 | node1 | skipping: [node1] 2026-08-18 14:54:25.990975 | node1 | skipping: [node2] 2026-08-18 14:54:25.990981 | node1 | skipping: [node3] 2026-08-18 14:54:25.990987 | node1 | 2026-08-18 14:54:25.990993 | node1 | TASK [vexxhost.ceph.cephadm : Pre-pull the Ceph image] ************************* 2026-08-18 14:54:25.990998 | node1 | Tuesday 18 August 2026 14:54:25 +0000 (0:00:00.076) 0:12:00.812 ******** 2026-08-18 14:54:26.470576 | node1 | ok: [node1] 2026-08-18 14:54:26.470655 | node1 | ok: [node3] 2026-08-18 14:54:26.472091 | node1 | ok: [node2] 2026-08-18 14:54:26.472152 | node1 | 2026-08-18 14:54:26.472157 | node1 | TASK [vexxhost.ceph.cephadm : Pin CEPHADM_IMAGE for interactive login shells] *** 2026-08-18 14:54:26.472163 | node1 | Tuesday 18 August 2026 14:54:26 +0000 (0:00:00.476) 0:12:01.289 ******** 2026-08-18 14:54:27.069636 | node1 | ok: [node2] 2026-08-18 14:54:27.069701 | node1 | ok: [node1] 2026-08-18 14:54:27.069799 | node1 | ok: [node3] 2026-08-18 14:54:27.070188 | node1 | 2026-08-18 14:54:27.070230 | node1 | TASK [vexxhost.ceph.rgw : Locate the cluster admin host] *********************** 2026-08-18 14:54:27.070236 | node1 | Tuesday 18 August 2026 14:54:27 +0000 (0:00:00.602) 0:12:01.891 ******** 2026-08-18 14:54:27.885630 | node1 | ok: [node1] => (item=node1) 2026-08-18 14:54:27.885715 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 14:54:27.885730 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 14:54:27.885857 | node1 | 2026-08-18 14:54:27.885972 | node1 | TASK [vexxhost.ceph.rgw : Select the admin host] ******************************* 2026-08-18 14:54:27.886142 | node1 | Tuesday 18 August 2026 14:54:27 +0000 (0:00:00.816) 0:12:02.707 ******** 2026-08-18 14:54:27.981165 | node1 | ok: [node1] 2026-08-18 14:54:27.981226 | node1 | ok: [node2] 2026-08-18 14:54:27.981236 | node1 | ok: [node3] 2026-08-18 14:54:27.981246 | node1 | 2026-08-18 14:54:27.981256 | node1 | TASK [vexxhost.ceph.rgw : Set the admin mon config path] *********************** 2026-08-18 14:54:27.981265 | node1 | Tuesday 18 August 2026 14:54:27 +0000 (0:00:00.094) 0:12:02.802 ******** 2026-08-18 14:54:28.075194 | node1 | ok: [node1] 2026-08-18 14:54:28.075231 | node1 | ok: [node2] 2026-08-18 14:54:28.075241 | node1 | ok: [node3] 2026-08-18 14:54:28.075410 | node1 | 2026-08-18 14:54:28.075660 | node1 | TASK [Register this RGW host] ************************************************** 2026-08-18 14:54:28.075906 | node1 | Tuesday 18 August 2026 14:54:28 +0000 (0:00:00.095) 0:12:02.897 ******** 2026-08-18 14:54:28.201762 | node1 | included: vexxhost.ceph.cephadm_host for node1, node2, node3 2026-08-18 14:54:28.201960 | node1 | 2026-08-18 14:54:28.202240 | node1 | TASK [vexxhost.ceph.cephadm_host : Get the cephadm public key] ***************** 2026-08-18 14:54:28.202487 | node1 | Tuesday 18 August 2026 14:54:28 +0000 (0:00:00.126) 0:12:03.024 ******** 2026-08-18 14:54:30.011092 | node1 | ok: [node1] 2026-08-18 14:54:30.011159 | node1 | 2026-08-18 14:54:30.011173 | node1 | TASK [vexxhost.ceph.cephadm_host : Resolve the cluster SSH public key to authorise] *** 2026-08-18 14:54:30.011186 | node1 | Tuesday 18 August 2026 14:54:30 +0000 (0:00:01.808) 0:12:04.832 ******** 2026-08-18 14:54:30.105373 | node1 | ok: [node1] 2026-08-18 14:54:30.106850 | node1 | ok: [node2] 2026-08-18 14:54:30.106877 | node1 | ok: [node3] 2026-08-18 14:54:30.106885 | node1 | 2026-08-18 14:54:30.106905 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert a cluster SSH key was resolved] ****** 2026-08-18 14:54:30.106926 | node1 | Tuesday 18 August 2026 14:54:30 +0000 (0:00:00.094) 0:12:04.926 ******** 2026-08-18 14:54:30.184825 | node1 | ok: [node1] => { 2026-08-18 14:54:30.185051 | node1 | "changed": false, 2026-08-18 14:54:30.185298 | node1 | "msg": "All assertions passed" 2026-08-18 14:54:30.185532 | node1 | } 2026-08-18 14:54:30.185772 | node1 | ok: [node2] => { 2026-08-18 14:54:30.186034 | node1 | "changed": false, 2026-08-18 14:54:30.186277 | node1 | "msg": "All assertions passed" 2026-08-18 14:54:30.186515 | node1 | } 2026-08-18 14:54:30.186812 | node1 | ok: [node3] => { 2026-08-18 14:54:30.187158 | node1 | "changed": false, 2026-08-18 14:54:30.187515 | node1 | "msg": "All assertions passed" 2026-08-18 14:54:30.187800 | node1 | } 2026-08-18 14:54:30.188129 | node1 | 2026-08-18 14:54:30.188477 | node1 | TASK [vexxhost.ceph.cephadm_host : Authorise the cluster SSH key for the cephadm user] *** 2026-08-18 14:54:30.188804 | node1 | Tuesday 18 August 2026 14:54:30 +0000 (0:00:00.080) 0:12:05.007 ******** 2026-08-18 14:54:30.544905 | node1 | ok: [node1] 2026-08-18 14:54:30.545138 | node1 | ok: [node2] 2026-08-18 14:54:30.545372 | node1 | ok: [node3] 2026-08-18 14:54:30.545588 | node1 | 2026-08-18 14:54:30.545846 | node1 | TASK [vexxhost.ceph.cephadm_host : Derive this host's cephadm labels from inventory membership] *** 2026-08-18 14:54:30.546128 | node1 | Tuesday 18 August 2026 14:54:30 +0000 (0:00:00.359) 0:12:05.366 ******** 2026-08-18 14:54:30.627656 | node1 | ok: [node1] 2026-08-18 14:54:30.627872 | node1 | ok: [node2] 2026-08-18 14:54:30.628159 | node1 | ok: [node3] 2026-08-18 14:54:30.628390 | node1 | 2026-08-18 14:54:30.628697 | node1 | TASK [vexxhost.ceph.cephadm_host : Find this host's addresses on the public network] *** 2026-08-18 14:54:30.628943 | node1 | Tuesday 18 August 2026 14:54:30 +0000 (0:00:00.083) 0:12:05.449 ******** 2026-08-18 14:54:30.755157 | node1 | ok: [node1] 2026-08-18 14:54:30.755390 | node1 | ok: [node2] 2026-08-18 14:54:30.755653 | node1 | ok: [node3] 2026-08-18 14:54:30.755883 | node1 | 2026-08-18 14:54:30.756170 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert exactly one address matches the public network] *** 2026-08-18 14:54:30.756409 | node1 | Tuesday 18 August 2026 14:54:30 +0000 (0:00:00.127) 0:12:05.577 ******** 2026-08-18 14:54:30.841950 | node1 | ok: [node1] => { 2026-08-18 14:54:30.842178 | node1 | "changed": false, 2026-08-18 14:54:30.842454 | node1 | "msg": "All assertions passed" 2026-08-18 14:54:30.842694 | node1 | } 2026-08-18 14:54:30.843046 | node1 | ok: [node2] => { 2026-08-18 14:54:30.843322 | node1 | "changed": false, 2026-08-18 14:54:30.843625 | node1 | "msg": "All assertions passed" 2026-08-18 14:54:30.843850 | node1 | } 2026-08-18 14:54:30.844152 | node1 | ok: [node3] => { 2026-08-18 14:54:30.844376 | node1 | "changed": false, 2026-08-18 14:54:30.844629 | node1 | "msg": "All assertions passed" 2026-08-18 14:54:30.844833 | node1 | } 2026-08-18 14:54:30.845081 | node1 | 2026-08-18 14:54:30.845430 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host (labels)] ***************** 2026-08-18 14:54:30.845746 | node1 | Tuesday 18 August 2026 14:54:30 +0000 (0:00:00.086) 0:12:05.663 ******** 2026-08-18 14:54:32.666829 | node1 | ok: [node1] 2026-08-18 14:54:32.667081 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:54:32.667360 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:54:32.667585 | node1 | 2026-08-18 14:54:32.667840 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host with a CRUSH location] **** 2026-08-18 14:54:32.668131 | node1 | Tuesday 18 August 2026 14:54:32 +0000 (0:00:01.825) 0:12:07.489 ******** 2026-08-18 14:54:32.773747 | node1 | skipping: [node1] 2026-08-18 14:54:32.773990 | node1 | skipping: [node2] 2026-08-18 14:54:32.774294 | node1 | skipping: [node3] 2026-08-18 14:54:32.774515 | node1 | 2026-08-18 14:54:32.774773 | node1 | TASK [vexxhost.ceph.rgw : Configure RGW multisite (realm, zonegroup, zone)] **** 2026-08-18 14:54:32.775035 | node1 | Tuesday 18 August 2026 14:54:32 +0000 (0:00:00.107) 0:12:07.596 ******** 2026-08-18 14:54:32.837764 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/rgw/tasks/multisite.yml for node1 2026-08-18 14:54:32.838033 | node1 | 2026-08-18 14:54:32.838343 | node1 | TASK [vexxhost.ceph.rgw : Configure RGW realms] ******************************** 2026-08-18 14:54:32.838594 | node1 | Tuesday 18 August 2026 14:54:32 +0000 (0:00:00.063) 0:12:07.660 ******** 2026-08-18 14:54:36.514209 | node1 | changed: [node1] => (item=molecule) 2026-08-18 14:54:36.514284 | node1 | 2026-08-18 14:54:36.514335 | node1 | TASK [vexxhost.ceph.rgw : Configure RGW zonegroups] **************************** 2026-08-18 14:54:36.514655 | node1 | Tuesday 18 August 2026 14:54:36 +0000 (0:00:03.675) 0:12:11.335 ******** 2026-08-18 14:54:39.038209 | node1 | changed: [node1] => (item=molecule) 2026-08-18 14:54:39.038303 | node1 | 2026-08-18 14:54:39.038715 | node1 | TASK [vexxhost.ceph.rgw : Configure RGW zones] ********************************* 2026-08-18 14:54:39.038778 | node1 | Tuesday 18 August 2026 14:54:39 +0000 (0:00:02.524) 0:12:13.859 ******** 2026-08-18 14:54:41.610035 | node1 | changed: [node1] => (item=None) 2026-08-18 14:54:41.610117 | node1 | changed: [node1 -> {{ ceph_rgw_admin_host }}] 2026-08-18 14:54:41.610131 | node1 | 2026-08-18 14:54:41.610144 | node1 | TASK [vexxhost.ceph.rgw : Commit the RGW period] ******************************* 2026-08-18 14:54:41.610157 | node1 | Tuesday 18 August 2026 14:54:41 +0000 (0:00:02.571) 0:12:16.431 ******** 2026-08-18 14:54:43.385535 | node1 | changed: [node1] 2026-08-18 14:54:43.385618 | node1 | 2026-08-18 14:54:43.385678 | node1 | TASK [vexxhost.ceph.rgw : Apply the RGW service specs] ************************* 2026-08-18 14:54:43.385806 | node1 | Tuesday 18 August 2026 14:54:43 +0000 (0:00:01.775) 0:12:18.207 ******** 2026-08-18 14:54:43.459916 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/rgw/tasks/services.yml for node1 2026-08-18 14:54:43.459993 | node1 | 2026-08-18 14:54:43.460047 | node1 | TASK [vexxhost.ceph.rgw : Snapshot current RGW service specs] ****************** 2026-08-18 14:54:43.460066 | node1 | Tuesday 18 August 2026 14:54:43 +0000 (0:00:00.073) 0:12:18.281 ******** 2026-08-18 14:54:45.200693 | node1 | ok: [node1] 2026-08-18 14:54:45.200752 | node1 | 2026-08-18 14:54:45.200766 | node1 | TASK [vexxhost.ceph.rgw : Index current RGW specs by service name] ************* 2026-08-18 14:54:45.200779 | node1 | Tuesday 18 August 2026 14:54:45 +0000 (0:00:01.740) 0:12:20.021 ******** 2026-08-18 14:54:45.256090 | node1 | ok: [node1] 2026-08-18 14:54:45.256897 | node1 | 2026-08-18 14:54:45.256975 | node1 | TASK [vexxhost.ceph.rgw : Apply the RGW service specs] ************************* 2026-08-18 14:54:45.256990 | node1 | Tuesday 18 August 2026 14:54:45 +0000 (0:00:00.056) 0:12:20.077 ******** 2026-08-18 14:54:48.658559 | node1 | changed: [node1] => (item=molecule) 2026-08-18 14:54:48.658632 | node1 | 2026-08-18 14:54:48.658747 | node1 | TASK [vexxhost.ceph.rgw : Redeploy RGW services whose redeploy-triggering keys changed] *** 2026-08-18 14:54:48.658856 | node1 | Tuesday 18 August 2026 14:54:48 +0000 (0:00:03.401) 0:12:23.479 ******** 2026-08-18 14:54:50.466542 | node1 | failed: [node1] (item=rgw.molecule) => {"ansible_loop_var": "item", "changed": true, "cmd": ["cephadm", "shell", "--fsid", "1dff0e0f-3c44-48da-81cd-4f3c6e8722b2", "--config", "/var/lib/ceph/1dff0e0f-3c44-48da-81cd-4f3c6e8722b2/mon.node1/config", "--", "ceph", "orch", "redeploy", "rgw.molecule"], "delta": "0:00:01.539828", "end": "2026-08-18 14:54:50.429688", "item": {"count_per_host": 1, "id": "molecule", "networks": ["0.0.0.0/0"], "spec": {"generate_cert": true, "rgw_frontend_port": 8080, "rgw_realm": "molecule", "rgw_zone": "molecule-1", "rgw_zonegroup": "molecule", "ssl": true}}, "msg": "non-zero return code", "rc": 22, "start": "2026-08-18 14:54:48.889860", "stderr": "Error EINVAL: No daemons exist under service name \"rgw.molecule\". View currently running services using \"ceph orch ls\"", "stderr_lines": ["Error EINVAL: No daemons exist under service name \"rgw.molecule\". View currently running services using \"ceph orch ls\""], "stdout": "", "stdout_lines": []} 2026-08-18 14:54:50.486603 | node1 | 2026-08-18 14:54:50.486734 | node1 | TASK [vexxhost.ceph.rgw : Remove the default zone / zonegroup / pools] ********* 2026-08-18 14:54:50.486854 | node1 | Tuesday 18 August 2026 14:54:50 +0000 (0:00:01.829) 0:12:25.308 ******** 2026-08-18 14:54:50.562527 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/rgw/tasks/cleanup.yml for node2 2026-08-18 14:54:50.562573 | node1 | 2026-08-18 14:54:50.562651 | node1 | TASK [vexxhost.ceph.rgw : List RGW zones] ************************************** 2026-08-18 14:54:50.562765 | node1 | Tuesday 18 August 2026 14:54:50 +0000 (0:00:00.074) 0:12:25.383 ******** 2026-08-18 14:54:52.712525 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:54:52.712615 | node1 | 2026-08-18 14:54:52.712773 | node1 | TASK [vexxhost.ceph.rgw : Remove the default zonegroup] ************************ 2026-08-18 14:54:52.712950 | node1 | Tuesday 18 August 2026 14:54:52 +0000 (0:00:02.151) 0:12:27.535 ******** 2026-08-18 14:54:52.762269 | node1 | skipping: [node2] 2026-08-18 14:54:52.762368 | node1 | 2026-08-18 14:54:52.762480 | node1 | TASK [vexxhost.ceph.rgw : Remove the default zone] ***************************** 2026-08-18 14:54:52.762597 | node1 | Tuesday 18 August 2026 14:54:52 +0000 (0:00:00.050) 0:12:27.585 ******** 2026-08-18 14:54:52.813247 | node1 | skipping: [node2] 2026-08-18 14:54:52.813347 | node1 | 2026-08-18 14:54:52.813460 | node1 | TASK [vexxhost.ceph.rgw : Get the current pool deletion setting] *************** 2026-08-18 14:54:52.813572 | node1 | Tuesday 18 August 2026 14:54:52 +0000 (0:00:00.049) 0:12:27.635 ******** 2026-08-18 14:54:52.866081 | node1 | skipping: [node2] 2026-08-18 14:54:52.866117 | node1 | 2026-08-18 14:54:52.866130 | node1 | TASK [vexxhost.ceph.rgw : Allow pool deletion] ********************************* 2026-08-18 14:54:52.866142 | node1 | Tuesday 18 August 2026 14:54:52 +0000 (0:00:00.053) 0:12:27.688 ******** 2026-08-18 14:54:52.930277 | node1 | skipping: [node2] 2026-08-18 14:54:52.930371 | node1 | 2026-08-18 14:54:52.930495 | node1 | TASK [vexxhost.ceph.rgw : Remove the default RGW pools] ************************ 2026-08-18 14:54:52.930609 | node1 | Tuesday 18 August 2026 14:54:52 +0000 (0:00:00.062) 0:12:27.751 ******** 2026-08-18 14:54:53.009708 | node1 | skipping: [node2] => (item=default.rgw.control) 2026-08-18 14:54:53.009823 | node1 | skipping: [node2] => (item=default.rgw.log) 2026-08-18 14:54:53.028176 | node1 | skipping: [node2] => (item=default.rgw.meta) 2026-08-18 14:54:53.028408 | node1 | skipping: [node2] 2026-08-18 14:54:53.028516 | node1 | 2026-08-18 14:54:53.028633 | node1 | TASK [vexxhost.ceph.rgw : Restore the configured pool deletion setting] ******** 2026-08-18 14:54:53.028741 | node1 | Tuesday 18 August 2026 14:54:53 +0000 (0:00:00.080) 0:12:27.832 ******** 2026-08-18 14:54:53.068396 | node1 | skipping: [node2] 2026-08-18 14:54:53.068531 | node1 | 2026-08-18 14:54:53.068695 | node1 | TASK [vexxhost.ceph.rgw : Remove the temporary pool deletion setting] ********** 2026-08-18 14:54:53.068866 | node1 | Tuesday 18 August 2026 14:54:53 +0000 (0:00:00.058) 0:12:27.890 ******** 2026-08-18 14:54:53.170030 | node1 | skipping: [node2] 2026-08-18 14:54:53.170321 | node1 | 2026-08-18 14:54:53.170589 | node1 | PLAY [Deploy Ceph monitoring stack] ******************************************** 2026-08-18 14:54:53.170823 | node1 | 2026-08-18 14:54:53.171132 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:54:53.171403 | node1 | Tuesday 18 August 2026 14:54:53 +0000 (0:00:00.101) 0:12:27.991 ******** 2026-08-18 14:54:54.216439 | node1 | ok: [node2] 2026-08-18 14:54:54.216565 | node1 | ok: [node3] 2026-08-18 14:54:54.216711 | node1 | 2026-08-18 14:54:54.216893 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:54:54.217095 | node1 | Tuesday 18 August 2026 14:54:54 +0000 (0:00:01.047) 0:12:29.038 ******** 2026-08-18 14:54:54.513691 | node1 | ok: [node2] 2026-08-18 14:54:54.513879 | node1 | ok: [node3] 2026-08-18 14:54:54.514156 | node1 | 2026-08-18 14:54:54.514395 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:54:54.514658 | node1 | Tuesday 18 August 2026 14:54:54 +0000 (0:00:00.297) 0:12:29.336 ******** 2026-08-18 14:54:54.648829 | node1 | skipping: [node2] 2026-08-18 14:54:54.649091 | node1 | skipping: [node3] 2026-08-18 14:54:54.649309 | node1 | 2026-08-18 14:54:54.649576 | node1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-08-18 14:54:54.649802 | node1 | Tuesday 18 August 2026 14:54:54 +0000 (0:00:00.135) 0:12:29.471 ******** 2026-08-18 14:54:54.934348 | node1 | ok: [node2] 2026-08-18 14:54:54.934555 | node1 | ok: [node3] 2026-08-18 14:54:54.934793 | node1 | 2026-08-18 14:54:54.935085 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:54:54.935402 | node1 | Tuesday 18 August 2026 14:54:54 +0000 (0:00:00.285) 0:12:29.756 ******** 2026-08-18 14:54:55.014394 | node1 | ok: [node2] => { 2026-08-18 14:54:55.014663 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:54:55.014903 | node1 | } 2026-08-18 14:54:55.015229 | node1 | ok: [node3] => { 2026-08-18 14:54:55.015494 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:54:55.015726 | node1 | } 2026-08-18 14:54:55.015953 | node1 | 2026-08-18 14:54:55.016242 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:54:55.016493 | node1 | Tuesday 18 August 2026 14:54:55 +0000 (0:00:00.080) 0:12:29.837 ******** 2026-08-18 14:54:55.428138 | node1 | ok: [node2] 2026-08-18 14:54:55.428216 | node1 | ok: [node3] 2026-08-18 14:54:55.428235 | node1 | 2026-08-18 14:54:55.428523 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:54:55.428771 | node1 | Tuesday 18 August 2026 14:54:55 +0000 (0:00:00.413) 0:12:30.250 ******** 2026-08-18 14:54:55.513521 | node1 | skipping: [node2] 2026-08-18 14:54:55.513610 | node1 | skipping: [node3] 2026-08-18 14:54:55.513708 | node1 | 2026-08-18 14:54:55.513831 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:54:55.513955 | node1 | Tuesday 18 August 2026 14:54:55 +0000 (0:00:00.085) 0:12:30.336 ******** 2026-08-18 14:54:55.596894 | node1 | skipping: [node2] 2026-08-18 14:54:55.596957 | node1 | skipping: [node3] 2026-08-18 14:54:55.596968 | node1 | 2026-08-18 14:54:55.596977 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:54:55.596987 | node1 | Tuesday 18 August 2026 14:54:55 +0000 (0:00:00.081) 0:12:30.417 ******** 2026-08-18 14:54:55.891580 | node1 | ok: [node2] 2026-08-18 14:54:55.891654 | node1 | ok: [node3] 2026-08-18 14:54:55.891756 | node1 | 2026-08-18 14:54:55.891869 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:54:55.891986 | node1 | Tuesday 18 August 2026 14:54:55 +0000 (0:00:00.295) 0:12:30.713 ******** 2026-08-18 14:54:56.870993 | node1 | ok: [node2] 2026-08-18 14:54:56.871085 | node1 | ok: [node3] 2026-08-18 14:54:56.871095 | node1 | 2026-08-18 14:54:56.871118 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:54:56.871128 | node1 | Tuesday 18 August 2026 14:54:56 +0000 (0:00:00.979) 0:12:31.692 ******** 2026-08-18 14:54:56.976238 | node1 | ok: [node2] => { 2026-08-18 14:54:56.976284 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:54:56.976291 | node1 | } 2026-08-18 14:54:56.976405 | node1 | ok: [node3] => { 2026-08-18 14:54:56.976586 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:54:56.977080 | node1 | } 2026-08-18 14:54:56.977096 | node1 | 2026-08-18 14:54:56.977104 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:54:56.977219 | node1 | Tuesday 18 August 2026 14:54:56 +0000 (0:00:00.105) 0:12:31.798 ******** 2026-08-18 14:54:57.470413 | node1 | ok: [node2] 2026-08-18 14:54:57.470512 | node1 | ok: [node3] 2026-08-18 14:54:57.470891 | node1 | 2026-08-18 14:54:57.470901 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:54:57.471052 | node1 | Tuesday 18 August 2026 14:54:57 +0000 (0:00:00.481) 0:12:32.279 ******** 2026-08-18 14:54:59.643112 | node1 | ok: [node2] 2026-08-18 14:54:59.643347 | node1 | ok: [node3] 2026-08-18 14:54:59.643580 | node1 | 2026-08-18 14:54:59.643848 | node1 | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-08-18 14:54:59.644142 | node1 | Tuesday 18 August 2026 14:54:59 +0000 (0:00:02.185) 0:12:34.465 ******** 2026-08-18 14:54:59.695879 | node1 | skipping: [node2] 2026-08-18 14:54:59.696120 | node1 | skipping: [node3] 2026-08-18 14:54:59.696334 | node1 | 2026-08-18 14:54:59.696584 | node1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-08-18 14:54:59.696913 | node1 | Tuesday 18 August 2026 14:54:59 +0000 (0:00:00.053) 0:12:34.518 ******** 2026-08-18 14:54:59.750970 | node1 | skipping: [node2] 2026-08-18 14:54:59.751184 | node1 | skipping: [node3] 2026-08-18 14:54:59.751403 | node1 | 2026-08-18 14:54:59.751652 | node1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-08-18 14:54:59.751897 | node1 | Tuesday 18 August 2026 14:54:59 +0000 (0:00:00.054) 0:12:34.573 ******** 2026-08-18 14:54:59.807797 | node1 | skipping: [node2] 2026-08-18 14:54:59.807989 | node1 | skipping: [node3] 2026-08-18 14:54:59.808241 | node1 | 2026-08-18 14:54:59.808486 | node1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-08-18 14:54:59.808729 | node1 | Tuesday 18 August 2026 14:54:59 +0000 (0:00:00.056) 0:12:34.629 ******** 2026-08-18 14:55:00.769930 | node1 | ok: [node2] 2026-08-18 14:55:00.770175 | node1 | ok: [node3] 2026-08-18 14:55:00.770398 | node1 | 2026-08-18 14:55:00.770661 | node1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-08-18 14:55:00.770908 | node1 | Tuesday 18 August 2026 14:55:00 +0000 (0:00:00.962) 0:12:35.592 ******** 2026-08-18 14:55:01.361363 | node1 | ok: [node2] 2026-08-18 14:55:01.361424 | node1 | ok: [node3] 2026-08-18 14:55:01.361535 | node1 | 2026-08-18 14:55:01.361648 | node1 | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-08-18 14:55:01.361761 | node1 | Tuesday 18 August 2026 14:55:01 +0000 (0:00:00.591) 0:12:36.183 ******** 2026-08-18 14:55:02.570078 | node1 | ok: [node2] => (item={'path': '/etc/containerd'}) 2026-08-18 14:55:02.570138 | node1 | ok: [node3] => (item={'path': '/etc/containerd'}) 2026-08-18 14:55:02.570151 | node1 | ok: [node2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:55:02.570164 | node1 | ok: [node3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:55:02.570175 | node1 | ok: [node2] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:55:02.570186 | node1 | ok: [node3] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:55:02.570198 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:55:02.570210 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:55:02.570221 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:55:02.570233 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:55:02.570244 | node1 | 2026-08-18 14:55:02.570257 | node1 | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-08-18 14:55:02.570268 | node1 | Tuesday 18 August 2026 14:55:02 +0000 (0:00:01.205) 0:12:37.389 ******** 2026-08-18 14:55:03.306238 | node1 | ok: [node2] 2026-08-18 14:55:03.306601 | node1 | ok: [node3] 2026-08-18 14:55:03.306890 | node1 | 2026-08-18 14:55:03.307273 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:55:03.307593 | node1 | Tuesday 18 August 2026 14:55:03 +0000 (0:00:00.725) 0:12:38.115 ******** 2026-08-18 14:55:03.307877 | node1 | 2026-08-18 14:55:03.308217 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:55:03.308863 | node1 | Tuesday 18 August 2026 14:55:03 +0000 (0:00:00.007) 0:12:38.122 ******** 2026-08-18 14:55:03.309659 | node1 | 2026-08-18 14:55:03.310793 | node1 | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-08-18 14:55:03.311704 | node1 | Tuesday 18 August 2026 14:55:03 +0000 (0:00:00.006) 0:12:38.129 ******** 2026-08-18 14:55:03.802559 | node1 | ok: [node2] 2026-08-18 14:55:03.802978 | node1 | ok: [node3] 2026-08-18 14:55:03.803346 | node1 | 2026-08-18 14:55:03.803683 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:55:03.804060 | node1 | Tuesday 18 August 2026 14:55:03 +0000 (0:00:00.495) 0:12:38.624 ******** 2026-08-18 14:55:04.120516 | node1 | ok: [node2] 2026-08-18 14:55:04.120893 | node1 | ok: [node3] 2026-08-18 14:55:04.121253 | node1 | 2026-08-18 14:55:04.121571 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:55:04.121877 | node1 | Tuesday 18 August 2026 14:55:04 +0000 (0:00:00.317) 0:12:38.942 ******** 2026-08-18 14:55:04.208780 | node1 | ok: [node2] => { 2026-08-18 14:55:04.209068 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:55:04.209606 | node1 | } 2026-08-18 14:55:04.209813 | node1 | ok: [node3] => { 2026-08-18 14:55:04.210046 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:55:04.210253 | node1 | } 2026-08-18 14:55:04.210437 | node1 | 2026-08-18 14:55:04.210697 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:55:04.210905 | node1 | Tuesday 18 August 2026 14:55:04 +0000 (0:00:00.088) 0:12:39.031 ******** 2026-08-18 14:55:04.653955 | node1 | ok: [node2] 2026-08-18 14:55:04.654367 | node1 | ok: [node3] 2026-08-18 14:55:04.654705 | node1 | 2026-08-18 14:55:04.655056 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:55:04.655387 | node1 | Tuesday 18 August 2026 14:55:04 +0000 (0:00:00.444) 0:12:39.476 ******** 2026-08-18 14:55:08.030578 | node1 | ok: [node2] 2026-08-18 14:55:08.030804 | node1 | ok: [node3] 2026-08-18 14:55:08.031059 | node1 | 2026-08-18 14:55:08.031326 | node1 | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-08-18 14:55:08.031596 | node1 | Tuesday 18 August 2026 14:55:08 +0000 (0:00:03.376) 0:12:42.852 ******** 2026-08-18 14:55:08.962529 | node1 | ok: [node2] 2026-08-18 14:55:08.962702 | node1 | ok: [node3] 2026-08-18 14:55:08.962929 | node1 | 2026-08-18 14:55:08.963231 | node1 | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-08-18 14:55:08.963475 | node1 | Tuesday 18 August 2026 14:55:08 +0000 (0:00:00.931) 0:12:43.784 ******** 2026-08-18 14:55:09.231926 | node1 | ok: [node2] 2026-08-18 14:55:09.232153 | node1 | ok: [node3] 2026-08-18 14:55:09.232512 | node1 | 2026-08-18 14:55:09.232788 | node1 | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-08-18 14:55:09.233102 | node1 | Tuesday 18 August 2026 14:55:09 +0000 (0:00:00.270) 0:12:44.054 ******** 2026-08-18 14:55:09.821715 | node1 | ok: [node2] 2026-08-18 14:55:09.821824 | node1 | ok: [node3] 2026-08-18 14:55:09.822069 | node1 | 2026-08-18 14:55:09.822095 | node1 | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-08-18 14:55:09.822271 | node1 | Tuesday 18 August 2026 14:55:09 +0000 (0:00:00.588) 0:12:44.643 ******** 2026-08-18 14:55:10.546833 | node1 | ok: [node2] => (item={'path': '/etc/docker'}) 2026-08-18 14:55:10.549129 | node1 | ok: [node3] => (item={'path': '/etc/docker'}) 2026-08-18 14:55:10.549210 | node1 | ok: [node2] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:55:10.549224 | node1 | ok: [node3] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:55:10.549237 | node1 | ok: [node2] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:55:10.549248 | node1 | ok: [node3] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:55:10.549260 | node1 | 2026-08-18 14:55:10.549273 | node1 | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-08-18 14:55:10.549284 | node1 | Tuesday 18 August 2026 14:55:10 +0000 (0:00:00.725) 0:12:45.368 ******** 2026-08-18 14:55:11.117378 | node1 | ok: [node2] 2026-08-18 14:55:11.117556 | node1 | ok: [node3] 2026-08-18 14:55:11.117829 | node1 | 2026-08-18 14:55:11.118185 | node1 | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-08-18 14:55:11.118489 | node1 | Tuesday 18 August 2026 14:55:11 +0000 (0:00:00.570) 0:12:45.939 ******** 2026-08-18 14:55:11.702090 | node1 | ok: [node2] 2026-08-18 14:55:11.702338 | node1 | ok: [node3] 2026-08-18 14:55:11.702595 | node1 | 2026-08-18 14:55:11.702848 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:55:11.703173 | node1 | Tuesday 18 August 2026 14:55:11 +0000 (0:00:00.570) 0:12:46.509 ******** 2026-08-18 14:55:11.703432 | node1 | 2026-08-18 14:55:11.703707 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:55:11.703952 | node1 | Tuesday 18 August 2026 14:55:11 +0000 (0:00:00.007) 0:12:46.517 ******** 2026-08-18 14:55:11.704265 | node1 | 2026-08-18 14:55:11.704615 | node1 | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-08-18 14:55:11.705050 | node1 | Tuesday 18 August 2026 14:55:11 +0000 (0:00:00.006) 0:12:46.524 ******** 2026-08-18 14:55:12.180825 | node1 | ok: [node2] 2026-08-18 14:55:12.181071 | node1 | ok: [node3] 2026-08-18 14:55:12.181303 | node1 | 2026-08-18 14:55:12.181554 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:55:12.181801 | node1 | Tuesday 18 August 2026 14:55:12 +0000 (0:00:00.478) 0:12:47.003 ******** 2026-08-18 14:55:12.242255 | node1 | ok: [node2] => { 2026-08-18 14:55:12.242462 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:55:12.242709 | node1 | } 2026-08-18 14:55:12.243057 | node1 | ok: [node3] => { 2026-08-18 14:55:12.243346 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:55:12.243591 | node1 | } 2026-08-18 14:55:12.243804 | node1 | 2026-08-18 14:55:12.244109 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:55:12.244379 | node1 | Tuesday 18 August 2026 14:55:12 +0000 (0:00:00.061) 0:12:47.064 ******** 2026-08-18 14:55:12.631817 | node1 | ok: [node2] 2026-08-18 14:55:12.631876 | node1 | ok: [node3] 2026-08-18 14:55:12.631887 | node1 | 2026-08-18 14:55:12.631988 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:55:12.632274 | node1 | Tuesday 18 August 2026 14:55:12 +0000 (0:00:00.389) 0:12:47.453 ******** 2026-08-18 14:55:12.705245 | node1 | skipping: [node2] 2026-08-18 14:55:12.705454 | node1 | skipping: [node3] 2026-08-18 14:55:12.705706 | node1 | 2026-08-18 14:55:12.705964 | node1 | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-08-18 14:55:12.706251 | node1 | Tuesday 18 August 2026 14:55:12 +0000 (0:00:00.074) 0:12:47.527 ******** 2026-08-18 14:55:12.775210 | node1 | ok: [node2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:55:12.775478 | node1 | ok: [node3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:55:12.775707 | node1 | 2026-08-18 14:55:12.775961 | node1 | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-08-18 14:55:12.776243 | node1 | Tuesday 18 August 2026 14:55:12 +0000 (0:00:00.070) 0:12:47.597 ******** 2026-08-18 14:55:13.716991 | node1 | ok: [node2] 2026-08-18 14:55:13.717209 | node1 | ok: [node3] 2026-08-18 14:55:13.717440 | node1 | 2026-08-18 14:55:13.717699 | node1 | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-08-18 14:55:13.717992 | node1 | Tuesday 18 August 2026 14:55:13 +0000 (0:00:00.940) 0:12:48.538 ******** 2026-08-18 14:55:14.575235 | node1 | ok: [node2] => (item=systemd-timesyncd) 2026-08-18 14:55:14.575464 | node1 | ok: [node3] => (item=systemd-timesyncd) 2026-08-18 14:55:14.575720 | node1 | ok: [node2] => (item=ssh) 2026-08-18 14:55:14.576057 | node1 | ok: [node3] => (item=ssh) 2026-08-18 14:55:14.576366 | node1 | 2026-08-18 14:55:14.576622 | node1 | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-08-18 14:55:14.576871 | node1 | Tuesday 18 August 2026 14:55:14 +0000 (0:00:00.858) 0:12:49.396 ******** 2026-08-18 14:55:14.862064 | node1 | ok: [node2] 2026-08-18 14:55:14.862321 | node1 | ok: [node3] 2026-08-18 14:55:14.862580 | node1 | 2026-08-18 14:55:14.862841 | node1 | TASK [vexxhost.ceph.cephadm : Ensure the cephadm SSH user exists] ************** 2026-08-18 14:55:14.863123 | node1 | Tuesday 18 August 2026 14:55:14 +0000 (0:00:00.286) 0:12:49.682 ******** 2026-08-18 14:55:15.211984 | node1 | ok: [node2] 2026-08-18 14:55:15.212310 | node1 | ok: [node3] 2026-08-18 14:55:15.212580 | node1 | 2026-08-18 14:55:15.212894 | node1 | TASK [vexxhost.ceph.cephadm : Grant passwordless sudo to the cephadm SSH user] *** 2026-08-18 14:55:15.213279 | node1 | Tuesday 18 August 2026 14:55:15 +0000 (0:00:00.351) 0:12:50.034 ******** 2026-08-18 14:55:15.781038 | node1 | ok: [node2] 2026-08-18 14:55:15.781170 | node1 | ok: [node3] 2026-08-18 14:55:15.781280 | node1 | 2026-08-18 14:55:15.781547 | node1 | TASK [vexxhost.ceph.cephadm : Log in to the container registry] **************** 2026-08-18 14:55:15.781718 | node1 | Tuesday 18 August 2026 14:55:15 +0000 (0:00:00.568) 0:12:50.603 ******** 2026-08-18 14:55:15.843508 | node1 | skipping: [node2] 2026-08-18 14:55:15.843759 | node1 | skipping: [node3] 2026-08-18 14:55:15.843985 | node1 | 2026-08-18 14:55:15.844287 | node1 | TASK [vexxhost.ceph.cephadm : Pre-pull the Ceph image] ************************* 2026-08-18 14:55:15.844538 | node1 | Tuesday 18 August 2026 14:55:15 +0000 (0:00:00.062) 0:12:50.665 ******** 2026-08-18 14:55:16.311487 | node1 | ok: [node2] 2026-08-18 14:55:16.311690 | node1 | ok: [node3] 2026-08-18 14:55:16.311914 | node1 | 2026-08-18 14:55:16.312221 | node1 | TASK [vexxhost.ceph.cephadm : Pin CEPHADM_IMAGE for interactive login shells] *** 2026-08-18 14:55:16.312472 | node1 | Tuesday 18 August 2026 14:55:16 +0000 (0:00:00.467) 0:12:51.133 ******** 2026-08-18 14:55:16.895938 | node1 | ok: [node2] 2026-08-18 14:55:16.896202 | node1 | ok: [node3] 2026-08-18 14:55:16.896440 | node1 | 2026-08-18 14:55:16.896710 | node1 | TASK [vexxhost.ceph.monitoring : Locate the cluster admin host] **************** 2026-08-18 14:55:16.896962 | node1 | Tuesday 18 August 2026 14:55:16 +0000 (0:00:00.584) 0:12:51.717 ******** 2026-08-18 14:55:17.654793 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] => (item=node1) 2026-08-18 14:55:17.655066 | node1 | ok: [node2] => (item=node2) 2026-08-18 14:55:17.655338 | node1 | ok: [node2 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 14:55:17.655581 | node1 | 2026-08-18 14:55:17.655866 | node1 | TASK [vexxhost.ceph.monitoring : Select the admin host] ************************ 2026-08-18 14:55:17.656207 | node1 | Tuesday 18 August 2026 14:55:17 +0000 (0:00:00.759) 0:12:52.477 ******** 2026-08-18 14:55:17.726430 | node1 | ok: [node2] 2026-08-18 14:55:17.726658 | node1 | ok: [node3] 2026-08-18 14:55:17.726887 | node1 | 2026-08-18 14:55:17.727176 | node1 | TASK [vexxhost.ceph.monitoring : Set the admin mon config path] **************** 2026-08-18 14:55:17.727431 | node1 | Tuesday 18 August 2026 14:55:17 +0000 (0:00:00.072) 0:12:52.549 ******** 2026-08-18 14:55:17.793673 | node1 | ok: [node2] 2026-08-18 14:55:17.793859 | node1 | ok: [node3] 2026-08-18 14:55:17.794125 | node1 | 2026-08-18 14:55:17.794385 | node1 | TASK [vexxhost.ceph.monitoring : Pin monitoring container images] ************** 2026-08-18 14:55:17.794651 | node1 | Tuesday 18 August 2026 14:55:17 +0000 (0:00:00.067) 0:12:52.616 ******** 2026-08-18 14:55:17.830647 | node1 | skipping: [node2] 2026-08-18 14:55:17.830911 | node1 | 2026-08-18 14:55:17.831244 | node1 | TASK [vexxhost.ceph.monitoring : Apply the monitoring service specs] *********** 2026-08-18 14:55:17.831514 | node1 | Tuesday 18 August 2026 14:55:17 +0000 (0:00:00.036) 0:12:52.652 ******** 2026-08-18 14:55:21.250348 | node1 | changed: [node2 -> node1({{ nodepool.private_ipv4 }})] => (item=ceph-exporter) 2026-08-18 14:55:21.250576 | node1 | 2026-08-18 14:55:21.250882 | node1 | PLAY RECAP ********************************************************************* 2026-08-18 14:55:21.251203 | node1 | node1 : ok=204 changed=46 unreachable=0 failed=1 skipped=37 rescued=0 ignored=0 2026-08-18 14:55:21.251482 | node1 | node2 : ok=210 changed=28 unreachable=0 failed=0 skipped=53 rescued=0 ignored=0 2026-08-18 14:55:21.251761 | node1 | node3 : ok=206 changed=27 unreachable=0 failed=0 skipped=45 rescued=0 ignored=0 2026-08-18 14:55:21.251986 | node1 | 2026-08-18 14:55:21.252242 | node1 | 2026-08-18 14:55:21.252483 | node1 | TASKS RECAP ******************************************************************** 2026-08-18 14:55:21.252731 | node1 | Tuesday 18 August 2026 14:55:21 +0000 (0:00:03.420) 0:12:56.073 ******** 2026-08-18 14:55:21.252962 | node1 | =============================================================================== 2026-08-18 14:55:21.253244 | node1 | vexxhost.ceph.osd : Deploy OSDs on the explicitly listed devices ------ 354.54s 2026-08-18 14:55:21.253494 | node1 | vexxhost.ceph.mon : Bootstrap the cluster ------------------------------ 43.38s 2026-08-18 14:55:21.253739 | node1 | vexxhost.ceph.cephadm : Pre-pull the Ceph image ------------------------ 40.10s 2026-08-18 14:55:21.253973 | node1 | vexxhost.ceph.configure : Configure pools ------------------------------ 35.80s 2026-08-18 14:55:21.254273 | node1 | vexxhost.ceph.osd : Wait for explicitly requested OSD daemons to be running on this host -- 21.96s 2026-08-18 14:55:21.254642 | node1 | vexxhost.ceph.cephadm_host : Register the host (labels) ---------------- 21.09s 2026-08-18 14:55:21.254919 | node1 | vexxhost.ceph.configure : Apply central Ceph configuration ------------- 19.69s 2026-08-18 14:55:21.255203 | node1 | vexxhost.ceph.configure : Configure CRUSH rules ------------------------ 11.71s 2026-08-18 14:55:21.255475 | node1 | vexxhost.ceph.configure : Configure dashboard users -------------------- 10.59s 2026-08-18 14:55:21.255638 | node1 | vexxhost.ceph.mgr : Configure ceph-mgr modules ------------------------- 10.27s 2026-08-18 14:55:21.255792 | node1 | vexxhost.ceph.mgr : Apply the manager service spec ---------------------- 8.41s 2026-08-18 14:55:21.255940 | node1 | vexxhost.containers.containerd : Install AppArmor packages -------------- 7.89s 2026-08-18 14:55:21.256149 | node1 | vexxhost.ceph.configure : Reload the dashboard so it picks up the certificate --- 7.26s 2026-08-18 14:55:21.256285 | node1 | vexxhost.ceph.mon : Wait for this monitor's daemon to be present -------- 6.34s 2026-08-18 14:55:21.256443 | node1 | vexxhost.containers.download_artifact : Extract archive ----------------- 4.83s 2026-08-18 14:55:21.256592 | node1 | vexxhost.ceph.configure : Configure erasure-code profiles --------------- 4.00s 2026-08-18 14:55:21.256747 | node1 | vexxhost.ceph.rgw : Configure RGW realms -------------------------------- 3.68s 2026-08-18 14:55:21.256898 | node1 | vexxhost.ceph.monitoring : Apply the monitoring service specs ----------- 3.42s 2026-08-18 14:55:21.257090 | node1 | vexxhost.ceph.rgw : Apply the RGW service specs ------------------------- 3.40s 2026-08-18 14:55:21.257231 | node1 | vexxhost.containers.download_artifact : Extract archive ----------------- 3.38s 2026-08-18 14:55:21.381799 | node1 | CRITICAL Ansible return code was 2, command was: ansible-playbook --inventory /home/zuul/.ansible/tmp/molecule.kre9.default/inventory --skip-tags molecule-notest,notest --inventory=/home/zuul/src/github.com/vexxhost/ansible-collection-ceph/inventory.yaml /home/zuul/src/github.com/vexxhost/ansible-collection-ceph/extensions/molecule/default/converge.yml 2026-08-18 14:55:21.381978 | node1 | ERROR [default > converge] Executed: Failed 2026-08-18 14:55:21.382163 | node1 | ERROR Ansible return code was 2, command was: ansible-playbook --inventory /home/zuul/.ansible/tmp/molecule.kre9.default/inventory --skip-tags molecule-notest,notest --inventory=/home/zuul/src/github.com/vexxhost/ansible-collection-ceph/inventory.yaml /home/zuul/src/github.com/vexxhost/ansible-collection-ceph/extensions/molecule/default/converge.yml 2026-08-18 14:55:31.590050 | node1 | ERROR 2026-08-18 14:55:31.590348 | node1 | { 2026-08-18 14:55:31.590431 | node1 | "delta": "0:14:17.536264", 2026-08-18 14:55:31.590478 | node1 | "end": "2026-08-18 14:55:21.479750", 2026-08-18 14:55:31.590519 | node1 | "msg": "non-zero return code", 2026-08-18 14:55:31.590560 | node1 | "rc": 2, 2026-08-18 14:55:31.590599 | node1 | "start": "2026-08-18 14:41:03.943486" 2026-08-18 14:55:31.590642 | node1 | } failure 2026-08-18 14:55:31.617238 | 2026-08-18 14:55:31.617334 | PLAY RECAP 2026-08-18 14:55:31.617382 | node1 | ok: 3 changed: 2 unreachable: 0 failed: 1 skipped: 0 rescued: 0 ignored: 0 2026-08-18 14:55:31.617404 | 2026-08-18 14:55:31.774891 | RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/run.yaml@main] 2026-08-18 14:55:31.791528 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main] 2026-08-18 14:55:32.431486 | 2026-08-18 14:55:32.432029 | PLAY [all] 2026-08-18 14:55:32.445220 | 2026-08-18 14:55:32.445296 | TASK [fetch-output : Set log path for multiple nodes] 2026-08-18 14:55:32.495358 | node1 | ok 2026-08-18 14:55:32.500257 | node2 | ok 2026-08-18 14:55:32.515409 | node3 | ok 2026-08-18 14:55:32.520232 | 2026-08-18 14:55:32.520319 | TASK [fetch-output : Set log path for single node] 2026-08-18 14:55:32.556383 | node1 | skipping: Conditional result was False 2026-08-18 14:55:32.557009 | node2 | skipping: Conditional result was False 2026-08-18 14:55:32.567331 | node3 | skipping: Conditional result was False 2026-08-18 14:55:32.572902 | 2026-08-18 14:55:32.572969 | LOOP [fetch-output : Ensure local output dirs] 2026-08-18 14:55:32.917724 | node1 -> localhost | changed: "/var/lib/zuul/builds/be0cdab9cf3446448d16157e8a3bf679/work/logs/node1" 2026-08-18 14:55:32.919951 | node3 -> localhost | changed: "/var/lib/zuul/builds/be0cdab9cf3446448d16157e8a3bf679/work/logs/node3" 2026-08-18 14:55:32.921649 | node2 -> localhost | changed: "/var/lib/zuul/builds/be0cdab9cf3446448d16157e8a3bf679/work/logs/node2" 2026-08-18 14:55:33.131871 | node1 -> localhost | changed: "/var/lib/zuul/builds/be0cdab9cf3446448d16157e8a3bf679/work/artifacts" 2026-08-18 14:55:33.144674 | node2 -> localhost | ok: "/var/lib/zuul/builds/be0cdab9cf3446448d16157e8a3bf679/work/artifacts" 2026-08-18 14:55:33.147637 | node3 -> localhost | ok: "/var/lib/zuul/builds/be0cdab9cf3446448d16157e8a3bf679/work/artifacts" 2026-08-18 14:55:33.342518 | node1 -> localhost | changed: "/var/lib/zuul/builds/be0cdab9cf3446448d16157e8a3bf679/work/docs" 2026-08-18 14:55:33.343799 | node2 -> localhost | ok: "/var/lib/zuul/builds/be0cdab9cf3446448d16157e8a3bf679/work/docs" 2026-08-18 14:55:33.348548 | node3 -> localhost | ok: "/var/lib/zuul/builds/be0cdab9cf3446448d16157e8a3bf679/work/docs" 2026-08-18 14:55:33.367366 | 2026-08-18 14:55:33.367489 | LOOP [fetch-output : Collect logs, artifacts and docs] 2026-08-18 14:55:33.976486 | node3 | changed: .d..t...... ./ 2026-08-18 14:55:33.976766 | node3 | changed: All items complete 2026-08-18 14:55:33.976806 | 2026-08-18 14:55:33.984674 | node2 | changed: .d..t...... ./ 2026-08-18 14:55:33.989685 | node1 | changed: .d..t...... ./ 2026-08-18 14:55:34.444521 | node1 | ok 2026-08-18 14:55:34.450507 | node2 | changed: .d..t...... ./ 2026-08-18 14:55:34.455300 | node3 | ok 2026-08-18 14:55:34.920991 | node1 | changed: .d..t...... ./ 2026-08-18 14:55:34.927288 | node3 | ok 2026-08-18 14:55:34.932572 | node2 | ok 2026-08-18 14:55:34.962474 | 2026-08-18 14:55:34.962601 | LOOP [merge-output-to-logs : Move artifacts and docs to logs dir] 2026-08-18 14:55:35.318708 | node1 -> localhost | ok: Item: artifacts Runtime: 0:00:00.008072 2026-08-18 14:55:35.318946 | node1 -> localhost | changed: All items complete 2026-08-18 14:55:35.318994 | 2026-08-18 14:55:35.530210 | node1 -> localhost | ok: Item: docs Runtime: 0:00:00.006549 2026-08-18 14:55:35.555918 | 2026-08-18 14:55:35.556045 | PLAY [all] 2026-08-18 14:55:35.563099 | 2026-08-18 14:55:35.563162 | TASK [remove-build-sshkey : Remove the build SSH key from all nodes] 2026-08-18 14:55:36.043446 | node1 | changed 2026-08-18 14:55:36.243960 | node2 | changed 2026-08-18 14:55:36.283723 | node3 | changed 2026-08-18 14:55:36.300214 | 2026-08-18 14:55:36.300268 | PLAY RECAP 2026-08-18 14:55:36.300315 | node1 | ok: 5 changed: 4 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0 2026-08-18 14:55:36.300345 | node2 | ok: 4 changed: 3 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0 2026-08-18 14:55:36.300371 | node3 | ok: 4 changed: 3 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0 2026-08-18 14:55:36.300390 | 2026-08-18 14:55:36.432400 | POST-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main] 2026-08-18 14:55:36.447903 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post-logs.yaml@main] 2026-08-18 14:55:37.056632 | 2026-08-18 14:55:37.056753 | PLAY [localhost] 2026-08-18 14:55:37.066225 | 2026-08-18 14:55:37.066297 | TASK [Generate Zuul manifest] 2026-08-18 14:55:37.087037 | localhost | ok 2026-08-18 14:55:37.103560 | 2026-08-18 14:55:37.103635 | TASK [generate-zuul-manifest : Generate Zuul manifest] 2026-08-18 14:55:37.453013 | localhost | changed 2026-08-18 14:55:37.463589 | 2026-08-18 14:55:37.463664 | TASK [generate-zuul-manifest : Return Zuul manifest URL to Zuul] 2026-08-18 14:55:37.496596 | localhost | ok 2026-08-18 14:55:37.504924 | 2026-08-18 14:55:37.504985 | TASK [Upload logs] 2026-08-18 14:55:37.526231 | localhost | ok 2026-08-18 14:55:37.617993 | 2026-08-18 14:55:37.618121 | TASK [Set zuul-log-path fact] 2026-08-18 14:55:37.640158 | localhost | ok 2026-08-18 14:55:37.652470 | 2026-08-18 14:55:37.652535 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-08-18 14:55:37.683681 | localhost | ok 2026-08-18 14:55:37.692497 | 2026-08-18 14:55:37.692561 | TASK [upload-logs : Create log directories] 2026-08-18 14:55:38.052158 | localhost | changed 2026-08-18 14:55:38.058761 | 2026-08-18 14:55:38.058863 | TASK [upload-logs : Ensure logs are readable before uploading] 2026-08-18 14:55:38.409902 | localhost -> localhost | ok: Runtime: 0:00:00.005392 2026-08-18 14:55:38.415579 | 2026-08-18 14:55:38.415649 | TASK [upload-logs : Upload logs to log server] 2026-08-18 14:55:38.878312 | localhost | Output suppressed because no_log was given 2026-08-18 14:55:38.885513 | 2026-08-18 14:55:38.885610 | LOOP [upload-logs : Compress console log and json output] 2026-08-18 14:55:38.929371 | localhost | skipping: Conditional result was False 2026-08-18 14:55:38.936154 | localhost | skipping: Conditional result was False 2026-08-18 14:55:38.950690 | 2026-08-18 14:55:38.950898 | LOOP [upload-logs : Upload compressed console log and json output] 2026-08-18 14:55:38.992316 | localhost | skipping: Conditional result was False 2026-08-18 14:55:38.992651 | 2026-08-18 14:55:38.996323 | localhost | skipping: Conditional result was False 2026-08-18 14:55:39.006250 | 2026-08-18 14:55:39.006418 | LOOP [upload-logs : Upload console log and json output]