2026-08-18 14:40:36.026381 | Job console starting 2026-08-18 14:40:36.043054 | Updating git repos 2026-08-18 14:40:36.067983 | Cloning repos into workspace 2026-08-18 14:40:36.102671 | Restoring repo states 2026-08-18 14:40:36.110153 | Merging changes 2026-08-18 14:40:36.981121 | Checking out repos 2026-08-18 14:40:37.044947 | Preparing playbooks 2026-08-18 14:40:38.305106 | Running Ansible setup 2026-08-18 14:40:43.397717 | PRE-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main] 2026-08-18 14:40:44.034320 | 2026-08-18 14:40:44.034444 | PLAY [localhost] 2026-08-18 14:40:44.041775 | 2026-08-18 14:40:44.041845 | TASK [Gathering Facts] 2026-08-18 14:40:45.074333 | localhost | ok 2026-08-18 14:40:45.085804 | 2026-08-18 14:40:45.085953 | TASK [Setup log path fact] 2026-08-18 14:40:45.107480 | localhost | ok 2026-08-18 14:40:45.120474 | 2026-08-18 14:40:45.120560 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-08-18 14:40:45.150232 | localhost | ok 2026-08-18 14:40:45.158475 | 2026-08-18 14:40:45.158570 | TASK [emit-job-header : Print job information] 2026-08-18 14:40:45.198489 | # Job Information 2026-08-18 14:40:45.198655 | Ansible Version: 2.16.18 2026-08-18 14:40:45.198702 | Job: ansible-collection-ceph-molecule-ha-ubuntu-jammy 2026-08-18 14:40:45.198733 | Pipeline: check 2026-08-18 14:40:45.198762 | Executor: 2d72f0692154 2026-08-18 14:40:45.198790 | Triggered by: https://github.com/vexxhost/ansible-collection-ceph/pull/123 2026-08-18 14:40:45.198823 | Event ID: 5876e6e0-9b12-11f1-83bd-8c5895bbc7af 2026-08-18 14:40:45.202600 | 2026-08-18 14:40:45.202689 | LOOP [emit-job-header : Print node information] 2026-08-18 14:40:45.334452 | localhost | ok: 2026-08-18 14:40:45.334846 | localhost | # Node Information 2026-08-18 14:40:45.334890 | localhost | Inventory Hostname: node1 2026-08-18 14:40:45.334923 | localhost | Hostname: np0000190025 2026-08-18 14:40:45.334954 | localhost | Username: zuul 2026-08-18 14:40:45.334987 | localhost | Distro: Ubuntu 22.04 2026-08-18 14:40:45.335018 | localhost | Provider: yul1 2026-08-18 14:40:45.335047 | localhost | Region: ca-ymq-1 2026-08-18 14:40:45.335076 | localhost | Label: ubuntu-jammy 2026-08-18 14:40:45.335104 | localhost | Product Name: OpenStack Nova 2026-08-18 14:40:45.335133 | localhost | Interface IP: 199.204.45.227 2026-08-18 14:40:45.392342 | localhost | ok: 2026-08-18 14:40:45.392450 | localhost | # Node Information 2026-08-18 14:40:45.392506 | localhost | Inventory Hostname: node2 2026-08-18 14:40:45.392553 | localhost | Hostname: np0000190026 2026-08-18 14:40:45.392596 | localhost | Username: zuul 2026-08-18 14:40:45.392638 | localhost | Distro: Ubuntu 22.04 2026-08-18 14:40:45.392680 | localhost | Provider: yul1 2026-08-18 14:40:45.392721 | localhost | Region: ca-ymq-1 2026-08-18 14:40:45.392761 | localhost | Label: ubuntu-jammy 2026-08-18 14:40:45.392802 | localhost | Product Name: OpenStack Nova 2026-08-18 14:40:45.392843 | localhost | Interface IP: 199.204.45.72 2026-08-18 14:40:45.439215 | localhost | ok: 2026-08-18 14:40:45.439390 | localhost | # Node Information 2026-08-18 14:40:45.439549 | localhost | Inventory Hostname: node3 2026-08-18 14:40:45.439604 | localhost | Hostname: np0000190027 2026-08-18 14:40:45.439656 | localhost | Username: zuul 2026-08-18 14:40:45.439704 | localhost | Distro: Ubuntu 22.04 2026-08-18 14:40:45.439749 | localhost | Provider: yul1 2026-08-18 14:40:45.439793 | localhost | Region: ca-ymq-1 2026-08-18 14:40:45.439838 | localhost | Label: ubuntu-jammy 2026-08-18 14:40:45.439881 | localhost | Product Name: OpenStack Nova 2026-08-18 14:40:45.439925 | localhost | Interface IP: 199.204.45.116 2026-08-18 14:40:45.450975 | 2026-08-18 14:40:45.451112 | TASK [log-inventory : Ensure Zuul Ansible directory exists] 2026-08-18 14:40:45.832107 | localhost -> localhost | changed 2026-08-18 14:40:45.841823 | 2026-08-18 14:40:45.841891 | TASK [log-inventory : Copy ansible inventory to logs dir] 2026-08-18 14:40:46.732396 | localhost -> localhost | changed 2026-08-18 14:40:46.741232 | 2026-08-18 14:40:46.741308 | PLAY [all] 2026-08-18 14:40:46.750872 | 2026-08-18 14:40:46.750937 | TASK [add-build-sshkey : Check to see if ssh key was already created for this build] 2026-08-18 14:40:47.018235 | node1 -> localhost | ok 2026-08-18 14:40:47.029529 | 2026-08-18 14:40:47.029595 | TASK [add-build-sshkey : Create a new key in workspace based on build UUID] 2026-08-18 14:40:47.060136 | node1 | ok 2026-08-18 14:40:47.064341 | node2 | ok 2026-08-18 14:40:47.082999 | node3 | ok 2026-08-18 14:40:47.148325 | node1 | included: /var/lib/zuul/builds/80089b70c9bf4e9cbfe604215db5f29f/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml 2026-08-18 14:40:47.148426 | node2 | included: /var/lib/zuul/builds/80089b70c9bf4e9cbfe604215db5f29f/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml 2026-08-18 14:40:47.148452 | node3 | included: /var/lib/zuul/builds/80089b70c9bf4e9cbfe604215db5f29f/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml 2026-08-18 14:40:47.154214 | 2026-08-18 14:40:47.154272 | TASK [add-build-sshkey : Create Temp SSH key] 2026-08-18 14:40:48.095183 | node1 -> localhost | Generating public/private rsa key pair. 2026-08-18 14:40:48.095393 | node1 -> localhost | Your identification has been saved in /var/lib/zuul/builds/80089b70c9bf4e9cbfe604215db5f29f/work/80089b70c9bf4e9cbfe604215db5f29f_id_rsa 2026-08-18 14:40:48.095438 | node1 -> localhost | Your public key has been saved in /var/lib/zuul/builds/80089b70c9bf4e9cbfe604215db5f29f/work/80089b70c9bf4e9cbfe604215db5f29f_id_rsa.pub 2026-08-18 14:40:48.095467 | node1 -> localhost | The key fingerprint is: 2026-08-18 14:40:48.095495 | node1 -> localhost | SHA256:YRsh7TkQ+aVQ8ZS8t1vS+3TLRCQewp0W9uzMBii9/WQ zuul-build-sshkey 2026-08-18 14:40:48.095524 | node1 -> localhost | The key's randomart image is: 2026-08-18 14:40:48.095551 | node1 -> localhost | +---[RSA 3072]----+ 2026-08-18 14:40:48.095582 | node1 -> localhost | | o=+o.. o | 2026-08-18 14:40:48.095613 | node1 -> localhost | | +..+* + = | 2026-08-18 14:40:48.095643 | node1 -> localhost | | =+=.* B + | 2026-08-18 14:40:48.095670 | node1 -> localhost | | .*+o B O | 2026-08-18 14:40:48.095694 | node1 -> localhost | | S. o = E | 2026-08-18 14:40:48.095713 | node1 -> localhost | | o O | 2026-08-18 14:40:48.095733 | node1 -> localhost | | + +o| 2026-08-18 14:40:48.095753 | node1 -> localhost | | . +.o| 2026-08-18 14:40:48.095774 | node1 -> localhost | | +.| 2026-08-18 14:40:48.095794 | node1 -> localhost | +----[SHA256]-----+ 2026-08-18 14:40:48.095847 | node1 -> localhost | ok: Runtime: 0:00:00.556207 2026-08-18 14:40:48.102588 | 2026-08-18 14:40:48.102660 | TASK [add-build-sshkey : Remote setup ssh keys (linux)] 2026-08-18 14:40:48.138684 | node1 | ok 2026-08-18 14:40:48.143621 | node2 | ok 2026-08-18 14:40:48.163012 | node3 | ok 2026-08-18 14:40:48.181076 | node1 | included: /var/lib/zuul/builds/80089b70c9bf4e9cbfe604215db5f29f/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml 2026-08-18 14:40:48.181133 | node2 | included: /var/lib/zuul/builds/80089b70c9bf4e9cbfe604215db5f29f/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml 2026-08-18 14:40:48.181158 | node3 | included: /var/lib/zuul/builds/80089b70c9bf4e9cbfe604215db5f29f/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml 2026-08-18 14:40:48.194157 | 2026-08-18 14:40:48.194226 | TASK [add-build-sshkey : Remove previously added zuul-build-sshkey] 2026-08-18 14:40:48.222133 | node1 | skipping: Conditional result was False 2026-08-18 14:40:48.224423 | node2 | skipping: Conditional result was False 2026-08-18 14:40:48.234884 | node3 | skipping: Conditional result was False 2026-08-18 14:40:48.246318 | 2026-08-18 14:40:48.246382 | TASK [add-build-sshkey : Enable access via build key on all nodes] 2026-08-18 14:40:48.688878 | node2 | changed 2026-08-18 14:40:48.694098 | node3 | changed 2026-08-18 14:40:48.723681 | node1 | changed 2026-08-18 14:40:48.732268 | 2026-08-18 14:40:48.732350 | TASK [add-build-sshkey : Make sure user has a .ssh] 2026-08-18 14:40:48.918403 | node2 | ok 2026-08-18 14:40:48.924067 | node1 | ok 2026-08-18 14:40:48.940209 | node3 | ok 2026-08-18 14:40:48.948354 | 2026-08-18 14:40:48.948448 | TASK [add-build-sshkey : Install build private key as SSH key on all nodes] 2026-08-18 14:40:49.394650 | node2 | changed 2026-08-18 14:40:49.423604 | node3 | changed 2026-08-18 14:40:49.452244 | node1 | changed 2026-08-18 14:40:49.459739 | 2026-08-18 14:40:49.459809 | TASK [add-build-sshkey : Install build public key as SSH key on all nodes] 2026-08-18 14:40:49.911220 | node2 | changed 2026-08-18 14:40:49.926188 | node3 | changed 2026-08-18 14:40:49.955076 | node1 | changed 2026-08-18 14:40:49.962413 | 2026-08-18 14:40:49.962477 | TASK [add-build-sshkey : Remote setup ssh keys (windows)] 2026-08-18 14:40:49.990954 | node1 | skipping: Conditional result was False 2026-08-18 14:40:50.002605 | node2 | skipping: Conditional result was False 2026-08-18 14:40:50.013193 | node3 | skipping: Conditional result was False 2026-08-18 14:40:50.022187 | 2026-08-18 14:40:50.022279 | TASK [remove-zuul-sshkey : Remove master key from local agent] 2026-08-18 14:40:50.401488 | node1 -> localhost | changed 2026-08-18 14:40:50.423275 | 2026-08-18 14:40:50.423358 | TASK [add-build-sshkey : Add back temp key] 2026-08-18 14:40:50.690305 | node1 -> localhost | Identity added: /var/lib/zuul/builds/80089b70c9bf4e9cbfe604215db5f29f/work/80089b70c9bf4e9cbfe604215db5f29f_id_rsa (zuul-build-sshkey) 2026-08-18 14:40:50.690532 | node1 -> localhost | ok: Runtime: 0:00:00.011668 2026-08-18 14:40:50.697110 | 2026-08-18 14:40:50.697195 | TASK [add-build-sshkey : Verify we can still SSH to all nodes] 2026-08-18 14:40:50.992509 | node2 | ok 2026-08-18 14:40:51.009447 | node1 | ok 2026-08-18 14:40:51.016147 | node3 | ok 2026-08-18 14:40:51.020471 | 2026-08-18 14:40:51.020533 | TASK [add-build-sshkey : Verify we can still SSH to all nodes (windows)] 2026-08-18 14:40:51.047993 | node1 | skipping: Conditional result was False 2026-08-18 14:40:51.058932 | node2 | skipping: Conditional result was False 2026-08-18 14:40:51.059217 | node3 | skipping: Conditional result was False 2026-08-18 14:40:51.082962 | 2026-08-18 14:40:51.083031 | TASK [prepare-workspace : Start zuul_console daemon.] 2026-08-18 14:40:51.442785 | node2 | ok 2026-08-18 14:40:51.446912 | node3 | ok 2026-08-18 14:40:51.450573 | node1 | ok 2026-08-18 14:40:51.455528 | 2026-08-18 14:40:51.455590 | TASK [prepare-workspace : Synchronize src repos to workspace directory.] 2026-08-18 14:40:52.132109 | node3 | Output suppressed because no_log was given 2026-08-18 14:40:52.137910 | node1 | Output suppressed because no_log was given 2026-08-18 14:40:52.141891 | node2 | Output suppressed because no_log was given 2026-08-18 14:40:52.158025 | 2026-08-18 14:40:52.158089 | LOOP [ensure-output-dirs : Empty Zuul Output directories by removing them] 2026-08-18 14:40:52.336763 | node1 | ok: "logs" 2026-08-18 14:40:52.337087 | node1 | ok: All items complete 2026-08-18 14:40:52.337128 | 2026-08-18 14:40:52.343741 | node2 | ok: "logs" 2026-08-18 14:40:52.361408 | node3 | ok: "logs" 2026-08-18 14:40:52.490202 | node1 | ok: "artifacts" 2026-08-18 14:40:52.494745 | node2 | ok: "artifacts" 2026-08-18 14:40:52.523525 | node3 | ok: "artifacts" 2026-08-18 14:40:52.661043 | node1 | ok: "docs" 2026-08-18 14:40:52.680340 | node3 | ok: "docs" 2026-08-18 14:40:52.685046 | node2 | ok: "docs" 2026-08-18 14:40:52.700928 | 2026-08-18 14:40:52.701066 | LOOP [ensure-output-dirs : Ensure Zuul Output directories exist] 2026-08-18 14:40:52.892107 | node1 | changed: "logs" 2026-08-18 14:40:52.892408 | node1 | ok: All items complete 2026-08-18 14:40:52.954818 | 2026-08-18 14:40:52.963844 | node2 | changed: "logs" 2026-08-18 14:40:52.972268 | node3 | changed: "logs" 2026-08-18 14:40:53.050681 | node1 | changed: "artifacts" 2026-08-18 14:40:53.056726 | node2 | changed: "artifacts" 2026-08-18 14:40:53.062126 | node3 | changed: "artifacts" 2026-08-18 14:40:53.200098 | node2 | changed: "docs" 2026-08-18 14:40:53.216484 | node1 | changed: "docs" 2026-08-18 14:40:53.250511 | node3 | changed: "docs" 2026-08-18 14:40:53.250578 | node3 | changed: All items complete 2026-08-18 14:40:53.250610 | 2026-08-18 14:40:53.273320 | 2026-08-18 14:40:53.273424 | PLAY RECAP 2026-08-18 14:40:53.273468 | localhost | ok: 6 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-08-18 14:40:53.273497 | node1 | ok: 15 changed: 8 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0 2026-08-18 14:40:53.273522 | node2 | ok: 11 changed: 5 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0 2026-08-18 14:40:53.273545 | node3 | ok: 11 changed: 5 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0 2026-08-18 14:40:53.273563 | 2026-08-18 14:40:53.418767 | PRE-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main] 2026-08-18 14:40:53.427408 | PRE-RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main] 2026-08-18 14:40:54.061861 | 2026-08-18 14:40:54.061982 | PLAY [all] 2026-08-18 14:40:54.073618 | 2026-08-18 14:40:54.073709 | TASK [setup-uv : Extract archive] 2026-08-18 14:40:56.304739 | node1 | changed 2026-08-18 14:40:56.367524 | node2 | changed 2026-08-18 14:40:56.447189 | node3 | changed 2026-08-18 14:40:56.454882 | 2026-08-18 14:40:56.455005 | TASK [setup-uv : Print version] 2026-08-18 14:40:56.809981 | node2 | uv 0.8.13 2026-08-18 14:40:56.824028 | node1 | uv 0.8.13 2026-08-18 14:40:56.817921 | node3 | uv 0.8.13 2026-08-18 14:40:57.081808 | node2 | ok: Runtime: 0:00:00.011270 2026-08-18 14:40:57.090316 | node1 | ok: Runtime: 0:00:00.011321 2026-08-18 14:40:57.099065 | node3 | ok: Runtime: 0:00:00.011708 2026-08-18 14:40:57.115732 | 2026-08-18 14:40:57.115802 | TASK [Install Ansible collection siblings] 2026-08-18 14:40:57.457289 | node1 | ok: 2026-08-18 14:40:57.457599 | node1 | Skipping community.crypto: no checked-out sibling 2026-08-18 14:40:57.457764 | node1 | Skipping community.docker: no checked-out sibling 2026-08-18 14:40:57.457825 | node1 | Skipping community.general: no checked-out sibling 2026-08-18 14:40:57.457872 | node1 | Skipping ansible.utils: no checked-out sibling 2026-08-18 14:40:57.457919 | node1 | Skipping vexxhost.containers: no checked-out sibling 2026-08-18 14:40:57.461711 | 2026-08-18 14:40:57.461815 | PLAY RECAP 2026-08-18 14:40:57.461915 | node1 | ok: 3 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-08-18 14:40:57.461987 | node2 | ok: 2 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-08-18 14:40:57.462051 | node3 | ok: 2 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-08-18 14:40:57.462077 | 2026-08-18 14:40:57.620138 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main] 2026-08-18 14:40:57.635930 | RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/run.yaml@main] 2026-08-18 14:40:58.315641 | 2026-08-18 14:40:58.315787 | PLAY [all] 2026-08-18 14:40:58.328474 | 2026-08-18 14:40:58.328553 | LOOP [Check inventory directories] 2026-08-18 14:40:58.647139 | node1 | ok: "src/github.com/vexxhost/ansible-collection-ceph" 2026-08-18 14:40:58.789413 | node1 | ok: "src/github.com/vexxhost/ansible-collection-ceph/molecule/default/inventory" 2026-08-18 14:40:58.931099 | node1 | ok: "src/github.com/vexxhost/ansible-collection-ceph/extensions/molecule/default/inventory" 2026-08-18 14:40:58.943036 | 2026-08-18 14:40:58.943217 | LOOP [Copy inventory file for Zuul] 2026-08-18 14:40:59.699748 | node1 | changed: 2026-08-18 14:40:59.699917 | node1 | { 2026-08-18 14:40:59.699945 | node1 | "ansible_loop_var": "item", 2026-08-18 14:40:59.699967 | node1 | "changed": false, 2026-08-18 14:40:59.699986 | node1 | "failed": false, 2026-08-18 14:40:59.700008 | node1 | "invocation": { 2026-08-18 14:40:59.700027 | node1 | "module_args": { 2026-08-18 14:40:59.700055 | node1 | "checksum_algorithm": "sha1", 2026-08-18 14:40:59.700074 | node1 | "follow": true, 2026-08-18 14:40:59.700091 | node1 | "get_attributes": true, 2026-08-18 14:40:59.700110 | node1 | "get_checksum": true, 2026-08-18 14:40:59.700127 | node1 | "get_mime": true, 2026-08-18 14:40:59.700144 | node1 | "path": "src/github.com/vexxhost/ansible-collection-ceph" 2026-08-18 14:40:59.700161 | node1 | } 2026-08-18 14:40:59.700178 | node1 | }, 2026-08-18 14:40:59.700195 | node1 | "item": "src/github.com/vexxhost/ansible-collection-ceph", 2026-08-18 14:40:59.700212 | node1 | "stat": { 2026-08-18 14:40:59.700229 | node1 | "atime": 1787064052.019398, 2026-08-18 14:40:59.700246 | node1 | "attr_flags": "e", 2026-08-18 14:40:59.700263 | node1 | "attributes": [ 2026-08-18 14:40:59.700280 | node1 | "extents" 2026-08-18 14:40:59.700297 | node1 | ], 2026-08-18 14:40:59.700314 | node1 | "block_size": 4096, 2026-08-18 14:40:59.700331 | node1 | "blocks": 8, 2026-08-18 14:40:59.700347 | node1 | "charset": "unknown", 2026-08-18 14:40:59.700365 | node1 | "ctime": 1787064052.027398, 2026-08-18 14:40:59.700383 | node1 | "dev": 64513, 2026-08-18 14:40:59.700399 | node1 | "device_type": 0, 2026-08-18 14:40:59.700416 | node1 | "executable": true, 2026-08-18 14:40:59.700432 | node1 | "exists": true, 2026-08-18 14:40:59.700449 | node1 | "gid": 1000, 2026-08-18 14:40:59.700465 | node1 | "gr_name": "zuul", 2026-08-18 14:40:59.700482 | node1 | "inode": 1042461, 2026-08-18 14:40:59.700498 | node1 | "isblk": false, 2026-08-18 14:40:59.700516 | node1 | "ischr": false, 2026-08-18 14:40:59.700532 | node1 | "isdir": true, 2026-08-18 14:40:59.700549 | node1 | "isfifo": false, 2026-08-18 14:40:59.700565 | node1 | "isgid": false, 2026-08-18 14:40:59.700582 | node1 | "islnk": false, 2026-08-18 14:40:59.700598 | node1 | "isreg": false, 2026-08-18 14:40:59.700614 | node1 | "issock": false, 2026-08-18 14:40:59.700631 | node1 | "isuid": false, 2026-08-18 14:40:59.700647 | node1 | "mimetype": "unknown", 2026-08-18 14:40:59.700663 | node1 | "mode": "0755", 2026-08-18 14:40:59.700679 | node1 | "mtime": 1787064036.8783822, 2026-08-18 14:40:59.700695 | node1 | "nlink": 10, 2026-08-18 14:40:59.700711 | node1 | "path": "src/github.com/vexxhost/ansible-collection-ceph", 2026-08-18 14:40:59.700727 | node1 | "pw_name": "zuul", 2026-08-18 14:40:59.700744 | node1 | "readable": true, 2026-08-18 14:40:59.700761 | node1 | "rgrp": true, 2026-08-18 14:40:59.700777 | node1 | "roth": true, 2026-08-18 14:40:59.700794 | node1 | "rusr": true, 2026-08-18 14:40:59.700811 | node1 | "size": 4096, 2026-08-18 14:40:59.700829 | node1 | "uid": 1000, 2026-08-18 14:40:59.700845 | node1 | "version": "3150924479", 2026-08-18 14:40:59.700863 | node1 | "wgrp": false, 2026-08-18 14:40:59.700881 | node1 | "woth": false, 2026-08-18 14:40:59.700898 | node1 | "writeable": true, 2026-08-18 14:40:59.700915 | node1 | "wusr": true, 2026-08-18 14:40:59.700932 | node1 | "xgrp": true, 2026-08-18 14:40:59.700948 | node1 | "xoth": true, 2026-08-18 14:40:59.700985 | node1 | "xusr": true 2026-08-18 14:40:59.701003 | node1 | } 2026-08-18 14:40:59.701020 | node1 | } 2026-08-18 14:40:59.701054 | node1 | ok: All items complete 2026-08-18 14:40:59.701075 | 2026-08-18 14:40:59.702320 | node1 | skipping: Conditional result was False 2026-08-18 14:40:59.702899 | node1 | skipping: Conditional result was False 2026-08-18 14:40:59.718302 | 2026-08-18 14:40:59.718448 | LOOP [Switch "ansible_host" to private IP] 2026-08-18 14:41:00.054846 | node1 | changed: 3 replacements made 2026-08-18 14:41:00.056794 | node1 | skipping: Conditional result was False 2026-08-18 14:41:00.057701 | node1 | skipping: Conditional result was False 2026-08-18 14:41:00.075619 | 2026-08-18 14:41:00.075808 | TASK [Run Molecule scenario] 2026-08-18 14:41:00.867548 | node1 | Downloading cpython-3.13.7-linux-x86_64-gnu (download) (30.9MiB) 2026-08-18 14:41:01.468924 | node1 | Downloading cpython-3.13.7-linux-x86_64-gnu (download) 2026-08-18 14:41:01.639994 | node1 | Using CPython 3.13.7 2026-08-18 14:41:01.640065 | node1 | Creating virtual environment at: .venv 2026-08-18 14:41:01.698471 | node1 | Downloading cryptography (4.3MiB) 2026-08-18 14:41:01.701742 | node1 | Downloading netaddr (2.2MiB) 2026-08-18 14:41:01.702072 | node1 | Downloading pygments (1.2MiB) 2026-08-18 14:41:01.702796 | node1 | Downloading ansible-core (2.1MiB) 2026-08-18 14:41:01.877612 | node1 | Downloading netaddr 2026-08-18 14:41:01.887264 | node1 | Downloading cryptography 2026-08-18 14:41:01.941052 | node1 | Downloading pygments 2026-08-18 14:41:02.041391 | node1 | Downloading ansible-core 2026-08-18 14:41:02.068853 | node1 | Installed 42 packages in 27ms 2026-08-18 14:41:03.219347 | node1 | INFO Collection 'vexxhost.ceph' detected. 2026-08-18 14:41:03.219442 | node1 | INFO Scenarios will be used from 'extensions/molecule' 2026-08-18 14:41:03.803808 | node1 | INFO [default > discovery] scenario test matrix: dependency, cleanup, destroy, syntax, create, prepare, converge, idempotence, side_effect, verify, cleanup, destroy 2026-08-18 14:41:03.803924 | node1 | INFO [default > prerun] Performing prerun with role_name_check=0... 2026-08-18 14:41:30.898285 | node1 | INFO [default > dependency] Executing 2026-08-18 14:41:30.898631 | node1 | WARNING [default > dependency] Missing roles requirements file: requirements.yml 2026-08-18 14:41:30.899151 | node1 | WARNING [default > dependency] Missing collections requirements file: collections.yml 2026-08-18 14:41:30.899384 | node1 | WARNING [default > dependency] Executed: 2 missing (Remove from test_sequence to suppress) 2026-08-18 14:41:30.904707 | node1 | INFO [default > cleanup] Executing 2026-08-18 14:41:30.904950 | node1 | WARNING [default > cleanup] Executed: Missing playbook (Remove from test_sequence to suppress) 2026-08-18 14:41:30.909977 | node1 | INFO [default > destroy] Executing 2026-08-18 14:41:30.910020 | node1 | WARNING [default > destroy] Skipping, '--destroy=never' requested. 2026-08-18 14:41:30.910036 | node1 | INFO [default > destroy] Executed: Successful 2026-08-18 14:41:30.915123 | node1 | INFO [default > syntax] Executing 2026-08-18 14:41:32.077717 | node1 | 2026-08-18 14:41:32.077983 | node1 | playbook: /home/zuul/src/github.com/vexxhost/ansible-collection-ceph/extensions/molecule/default/converge.yml 2026-08-18 14:41:32.134748 | node1 | INFO [default > syntax] Executed: Successful 2026-08-18 14:41:32.144980 | node1 | INFO [default > create] Executing 2026-08-18 14:41:32.146787 | node1 | WARNING [default > create] Executed: Missing playbook (Remove from test_sequence to suppress) 2026-08-18 14:41:32.151652 | node1 | INFO [default > prepare] Executing 2026-08-18 14:41:32.827194 | node1 | 2026-08-18 14:41:32.827406 | node1 | PLAY [Prepare] ***************************************************************** 2026-08-18 14:41:32.827593 | node1 | 2026-08-18 14:41:32.827800 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:41:32.828000 | node1 | Tuesday 18 August 2026 14:41:32 +0000 (0:00:00.029) 0:00:00.029 ******** 2026-08-18 14:41:33.987074 | node1 | [WARNING]: Platform linux on host node1 is using the discovered Python 2026-08-18 14:41:33.987246 | node1 | interpreter at /usr/bin/python3.10, but future installation of another Python 2026-08-18 14:41:33.987441 | node1 | interpreter could change the meaning of that path. See 2026-08-18 14:41:33.987637 | node1 | https://docs.ansible.com/ansible- 2026-08-18 14:41:33.987838 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 14:41:33.989793 | node1 | [WARNING]: Platform linux on host node2 is using the discovered Python 2026-08-18 14:41:33.990011 | node1 | interpreter at /usr/bin/python3.10, but future installation of another Python 2026-08-18 14:41:33.990214 | node1 | interpreter could change the meaning of that path. See 2026-08-18 14:41:33.990409 | node1 | https://docs.ansible.com/ansible- 2026-08-18 14:41:33.990610 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 14:41:34.014960 | node1 | [WARNING]: Platform linux on host node3 is using the discovered Python 2026-08-18 14:41:34.015166 | node1 | interpreter at /usr/bin/python3.10, but future installation of another Python 2026-08-18 14:41:34.015375 | node1 | interpreter could change the meaning of that path. See 2026-08-18 14:41:34.015572 | node1 | https://docs.ansible.com/ansible- 2026-08-18 14:41:34.015774 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 14:41:34.024958 | node1 | ok: [node1] 2026-08-18 14:41:34.025187 | node1 | ok: [node2] 2026-08-18 14:41:34.025381 | node1 | ok: [node3] 2026-08-18 14:41:34.025567 | node1 | 2026-08-18 14:41:34.025780 | node1 | TASK [Configure short hostname] ************************************************ 2026-08-18 14:41:34.025983 | node1 | Tuesday 18 August 2026 14:41:34 +0000 (0:00:01.198) 0:00:01.227 ******** 2026-08-18 14:41:34.808328 | node1 | changed: [node1] 2026-08-18 14:41:34.808506 | node1 | changed: [node2] 2026-08-18 14:41:34.808700 | node1 | changed: [node3] 2026-08-18 14:41:34.808879 | node1 | 2026-08-18 14:41:34.809077 | node1 | TASK [Ensure hostname inside hosts file] *************************************** 2026-08-18 14:41:34.809306 | node1 | Tuesday 18 August 2026 14:41:34 +0000 (0:00:00.782) 0:00:02.010 ******** 2026-08-18 14:41:35.066683 | node1 | [WARNING]: Module remote_tmp /root/.ansible/tmp did not exist and was created 2026-08-18 14:41:35.067081 | node1 | with a mode of 0700, this may cause issues when running as another user. To 2026-08-18 14:41:35.067343 | node1 | avoid this, create the remote_tmp dir with the correct permissions manually 2026-08-18 14:41:35.084244 | node1 | changed: [node2] 2026-08-18 14:41:35.084494 | node1 | changed: [node1] 2026-08-18 14:41:35.084751 | node1 | changed: [node3] 2026-08-18 14:41:35.084980 | node1 | 2026-08-18 14:41:35.085348 | node1 | TASK [Update apt cache] ******************************************************** 2026-08-18 14:41:35.085645 | node1 | Tuesday 18 August 2026 14:41:35 +0000 (0:00:00.276) 0:00:02.286 ******** 2026-08-18 14:41:42.081045 | node1 | changed: [node2] 2026-08-18 14:41:42.081124 | node1 | changed: [node1] 2026-08-18 14:41:42.081221 | node1 | changed: [node3] 2026-08-18 14:41:42.081284 | node1 | 2026-08-18 14:41:42.081386 | node1 | PLAY [Create devices for Ceph] ************************************************* 2026-08-18 14:41:42.081468 | node1 | 2026-08-18 14:41:42.081562 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:41:42.081653 | node1 | Tuesday 18 August 2026 14:41:42 +0000 (0:00:06.997) 0:00:09.284 ******** 2026-08-18 14:41:42.792398 | node1 | ok: [node1] 2026-08-18 14:41:42.792471 | node1 | ok: [node2] 2026-08-18 14:41:42.792528 | node1 | ok: [node3] 2026-08-18 14:41:42.792604 | node1 | 2026-08-18 14:41:42.792693 | node1 | TASK [Install depedencies] ***************************************************** 2026-08-18 14:41:42.792788 | node1 | Tuesday 18 August 2026 14:41:42 +0000 (0:00:00.711) 0:00:09.995 ******** 2026-08-18 14:42:03.233301 | node1 | changed: [node2] 2026-08-18 14:42:03.233519 | node1 | changed: [node3] 2026-08-18 14:42:03.233754 | node1 | changed: [node1] 2026-08-18 14:42:03.234093 | node1 | 2026-08-18 14:42:03.234381 | node1 | TASK [Start up service] ******************************************************** 2026-08-18 14:42:03.234661 | node1 | Tuesday 18 August 2026 14:42:03 +0000 (0:00:20.440) 0:00:30.435 ******** 2026-08-18 14:42:03.846472 | node1 | ok: [node2] 2026-08-18 14:42:03.846546 | node1 | ok: [node3] 2026-08-18 14:42:03.846732 | node1 | ok: [node1] 2026-08-18 14:42:03.846848 | node1 | 2026-08-18 14:42:03.846986 | node1 | TASK [Generate lvm.conf] ******************************************************* 2026-08-18 14:42:03.847118 | node1 | Tuesday 18 August 2026 14:42:03 +0000 (0:00:00.613) 0:00:31.049 ******** 2026-08-18 14:42:04.165157 | node1 | ok: [node1] 2026-08-18 14:42:04.165210 | node1 | ok: [node2] 2026-08-18 14:42:04.165349 | node1 | ok: [node3] 2026-08-18 14:42:04.165425 | node1 | 2026-08-18 14:42:04.165510 | node1 | TASK [Write /etc/lvm/lvm.conf] ************************************************* 2026-08-18 14:42:04.165611 | node1 | Tuesday 18 August 2026 14:42:04 +0000 (0:00:00.319) 0:00:31.368 ******** 2026-08-18 14:42:04.878738 | node1 | changed: [node2] 2026-08-18 14:42:04.878926 | node1 | changed: [node3] 2026-08-18 14:42:04.879146 | node1 | changed: [node1] 2026-08-18 14:42:04.879353 | node1 | 2026-08-18 14:42:04.879579 | node1 | TASK [Get list of all loopback devices] **************************************** 2026-08-18 14:42:04.879806 | node1 | Tuesday 18 August 2026 14:42:04 +0000 (0:00:00.713) 0:00:32.081 ******** 2026-08-18 14:42:05.090419 | node1 | ok: [node1] 2026-08-18 14:42:05.090592 | node1 | ok: [node2] 2026-08-18 14:42:05.090818 | node1 | ok: [node3] 2026-08-18 14:42:05.091013 | node1 | 2026-08-18 14:42:05.091247 | node1 | TASK [Fail if there is any existing loopback devices] ************************** 2026-08-18 14:42:05.091479 | node1 | Tuesday 18 August 2026 14:42:05 +0000 (0:00:00.211) 0:00:32.292 ******** 2026-08-18 14:42:05.125973 | node1 | skipping: [node1] 2026-08-18 14:42:05.126199 | node1 | skipping: [node2] 2026-08-18 14:42:05.126411 | node1 | skipping: [node3] 2026-08-18 14:42:05.126606 | node1 | 2026-08-18 14:42:05.126814 | node1 | TASK [Create devices for Ceph] ************************************************* 2026-08-18 14:42:05.127027 | node1 | Tuesday 18 August 2026 14:42:05 +0000 (0:00:00.035) 0:00:32.328 ******** 2026-08-18 14:42:05.685349 | node1 | changed: [node1] => (item=osd0) 2026-08-18 14:42:05.685459 | node1 | changed: [node2] => (item=osd0) 2026-08-18 14:42:05.685676 | node1 | changed: [node3] => (item=osd0) 2026-08-18 14:42:05.685882 | node1 | changed: [node2] => (item=osd1) 2026-08-18 14:42:05.686080 | node1 | changed: [node1] => (item=osd1) 2026-08-18 14:42:05.686274 | node1 | changed: [node3] => (item=osd1) 2026-08-18 14:42:05.686468 | node1 | changed: [node2] => (item=osd2) 2026-08-18 14:42:05.686667 | node1 | changed: [node1] => (item=osd2) 2026-08-18 14:42:05.686854 | node1 | changed: [node3] => (item=osd2) 2026-08-18 14:42:05.687036 | node1 | 2026-08-18 14:42:05.687240 | node1 | TASK [Set permissions on loopback devices] ************************************* 2026-08-18 14:42:05.687446 | node1 | Tuesday 18 August 2026 14:42:05 +0000 (0:00:00.559) 0:00:32.887 ******** 2026-08-18 14:42:06.311729 | node1 | changed: [node2] => (item=osd0) 2026-08-18 14:42:06.311907 | node1 | changed: [node3] => (item=osd0) 2026-08-18 14:42:06.312123 | node1 | changed: [node1] => (item=osd0) 2026-08-18 14:42:06.312342 | node1 | changed: [node2] => (item=osd1) 2026-08-18 14:42:06.312544 | node1 | changed: [node3] => (item=osd1) 2026-08-18 14:42:06.312748 | node1 | changed: [node1] => (item=osd1) 2026-08-18 14:42:06.312950 | node1 | changed: [node2] => (item=osd2) 2026-08-18 14:42:06.313218 | node1 | changed: [node3] => (item=osd2) 2026-08-18 14:42:06.313420 | node1 | changed: [node1] => (item=osd2) 2026-08-18 14:42:06.313660 | node1 | 2026-08-18 14:42:06.313886 | node1 | TASK [Start loop devices] ****************************************************** 2026-08-18 14:42:06.314088 | node1 | Tuesday 18 August 2026 14:42:06 +0000 (0:00:00.626) 0:00:33.514 ******** 2026-08-18 14:42:07.046152 | node1 | changed: [node2] => (item=osd0) 2026-08-18 14:42:07.046305 | node1 | changed: [node1] => (item=osd0) 2026-08-18 14:42:07.046445 | node1 | changed: [node3] => (item=osd0) 2026-08-18 14:42:07.046583 | node1 | changed: [node2] => (item=osd1) 2026-08-18 14:42:07.046714 | node1 | changed: [node3] => (item=osd1) 2026-08-18 14:42:07.046839 | node1 | changed: [node1] => (item=osd1) 2026-08-18 14:42:07.046959 | node1 | changed: [node2] => (item=osd2) 2026-08-18 14:42:07.047083 | node1 | changed: [node3] => (item=osd2) 2026-08-18 14:42:07.047206 | node1 | changed: [node1] => (item=osd2) 2026-08-18 14:42:07.047326 | node1 | 2026-08-18 14:42:07.047455 | node1 | TASK [Create a volume group for each loop device] ****************************** 2026-08-18 14:42:07.047585 | node1 | Tuesday 18 August 2026 14:42:07 +0000 (0:00:00.734) 0:00:34.249 ******** 2026-08-18 14:42:10.241941 | node1 | changed: [node1] => (item=osd0) 2026-08-18 14:42:10.242156 | node1 | changed: [node3] => (item=osd0) 2026-08-18 14:42:10.242365 | node1 | changed: [node2] => (item=osd0) 2026-08-18 14:42:10.242567 | node1 | changed: [node2] => (item=osd1) 2026-08-18 14:42:10.242760 | node1 | changed: [node3] => (item=osd1) 2026-08-18 14:42:10.242955 | node1 | changed: [node1] => (item=osd1) 2026-08-18 14:42:10.243175 | node1 | changed: [node2] => (item=osd2) 2026-08-18 14:42:10.243370 | node1 | changed: [node1] => (item=osd2) 2026-08-18 14:42:10.243569 | node1 | changed: [node3] => (item=osd2) 2026-08-18 14:42:10.243774 | node1 | 2026-08-18 14:42:10.244054 | node1 | TASK [Create a logical volume for each loop device] **************************** 2026-08-18 14:42:10.244365 | node1 | Tuesday 18 August 2026 14:42:10 +0000 (0:00:03.195) 0:00:37.444 ******** 2026-08-18 14:42:12.399475 | node1 | changed: [node2] => (item=ceph-node2-osd0) 2026-08-18 14:42:12.399533 | node1 | changed: [node3] => (item=ceph-node3-osd0) 2026-08-18 14:42:12.399542 | node1 | changed: [node1] => (item=ceph-node1-osd0) 2026-08-18 14:42:12.399548 | node1 | changed: [node3] => (item=ceph-node3-osd1) 2026-08-18 14:42:12.399554 | node1 | changed: [node2] => (item=ceph-node2-osd1) 2026-08-18 14:42:12.399560 | node1 | changed: [node1] => (item=ceph-node1-osd1) 2026-08-18 14:42:12.399566 | node1 | changed: [node2] => (item=ceph-node2-osd2) 2026-08-18 14:42:12.399573 | node1 | changed: [node3] => (item=ceph-node3-osd2) 2026-08-18 14:42:12.399578 | node1 | changed: [node1] => (item=ceph-node1-osd2) 2026-08-18 14:42:12.399585 | node1 | 2026-08-18 14:42:12.399591 | node1 | PLAY RECAP ********************************************************************* 2026-08-18 14:42:12.399603 | node1 | node1 : ok=15 changed=10 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0 2026-08-18 14:42:12.401134 | node1 | node2 : ok=15 changed=10 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0 2026-08-18 14:42:12.401154 | node1 | node3 : ok=15 changed=10 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0 2026-08-18 14:42:12.401161 | node1 | 2026-08-18 14:42:12.401167 | node1 | 2026-08-18 14:42:12.401174 | node1 | TASKS RECAP ******************************************************************** 2026-08-18 14:42:12.401180 | node1 | Tuesday 18 August 2026 14:42:12 +0000 (0:00:02.156) 0:00:39.600 ******** 2026-08-18 14:42:12.401186 | node1 | =============================================================================== 2026-08-18 14:42:12.401192 | node1 | Install depedencies ---------------------------------------------------- 20.44s 2026-08-18 14:42:12.401198 | node1 | Update apt cache -------------------------------------------------------- 7.00s 2026-08-18 14:42:12.401204 | node1 | Create a volume group for each loop device ------------------------------ 3.20s 2026-08-18 14:42:12.401210 | node1 | Create a logical volume for each loop device ---------------------------- 2.16s 2026-08-18 14:42:12.401217 | node1 | Gathering Facts --------------------------------------------------------- 1.20s 2026-08-18 14:42:12.401223 | node1 | Configure short hostname ------------------------------------------------ 0.78s 2026-08-18 14:42:12.401229 | node1 | Start loop devices ------------------------------------------------------ 0.73s 2026-08-18 14:42:12.401235 | node1 | Write /etc/lvm/lvm.conf ------------------------------------------------- 0.71s 2026-08-18 14:42:12.401247 | node1 | Gathering Facts --------------------------------------------------------- 0.71s 2026-08-18 14:42:12.401254 | node1 | Set permissions on loopback devices ------------------------------------- 0.63s 2026-08-18 14:42:12.401260 | node1 | Start up service -------------------------------------------------------- 0.61s 2026-08-18 14:42:12.401269 | node1 | Create devices for Ceph ------------------------------------------------- 0.56s 2026-08-18 14:42:12.401653 | node1 | Generate lvm.conf ------------------------------------------------------- 0.32s 2026-08-18 14:42:12.401668 | node1 | Ensure hostname inside hosts file --------------------------------------- 0.28s 2026-08-18 14:42:12.401673 | node1 | Get list of all loopback devices ---------------------------------------- 0.21s 2026-08-18 14:42:12.401677 | node1 | Fail if there is any existing loopback devices -------------------------- 0.04s 2026-08-18 14:42:12.497791 | node1 | INFO [default > prepare] Executed: Successful 2026-08-18 14:42:12.505541 | node1 | INFO [default > converge] Executing 2026-08-18 14:42:13.601735 | node1 | 2026-08-18 14:42:13.602261 | node1 | PLAY [Deploy Ceph monitors & managers] ***************************************** 2026-08-18 14:42:13.602549 | node1 | 2026-08-18 14:42:13.602881 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:42:13.603214 | node1 | Tuesday 18 August 2026 14:42:13 +0000 (0:00:00.081) 0:00:00.081 ******** 2026-08-18 14:42:14.712554 | node1 | [WARNING]: Platform linux on host node2 is using the discovered Python 2026-08-18 14:42:14.712622 | node1 | interpreter at /usr/bin/python3.10, but future installation of another Python 2026-08-18 14:42:14.712652 | node1 | interpreter could change the meaning of that path. See 2026-08-18 14:42:14.712658 | node1 | https://docs.ansible.com/ansible- 2026-08-18 14:42:14.712671 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 14:42:14.751957 | node1 | [WARNING]: Platform linux on host node3 is using the discovered Python 2026-08-18 14:42:14.752174 | node1 | interpreter at /usr/bin/python3.10, but future installation of another Python 2026-08-18 14:42:14.752390 | node1 | interpreter could change the meaning of that path. See 2026-08-18 14:42:14.752608 | node1 | https://docs.ansible.com/ansible- 2026-08-18 14:42:14.752823 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 14:42:14.760679 | node1 | [WARNING]: Platform linux on host node1 is using the discovered Python 2026-08-18 14:42:14.760884 | node1 | interpreter at /usr/bin/python3.10, but future installation of another Python 2026-08-18 14:42:14.761081 | node1 | interpreter could change the meaning of that path. See 2026-08-18 14:42:14.761330 | node1 | https://docs.ansible.com/ansible- 2026-08-18 14:42:14.761532 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 14:42:14.813267 | node1 | ok: [node2] 2026-08-18 14:42:14.813478 | node1 | ok: [node3] 2026-08-18 14:42:14.813691 | node1 | ok: [node1] 2026-08-18 14:42:14.813897 | node1 | 2026-08-18 14:42:14.814129 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:42:14.814372 | node1 | Tuesday 18 August 2026 14:42:14 +0000 (0:00:01.211) 0:00:01.293 ******** 2026-08-18 14:42:15.115570 | node1 | ok: [node2] 2026-08-18 14:42:15.115655 | node1 | ok: [node1] 2026-08-18 14:42:15.115753 | node1 | ok: [node3] 2026-08-18 14:42:15.115865 | node1 | 2026-08-18 14:42:15.115997 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:42:15.116123 | node1 | Tuesday 18 August 2026 14:42:15 +0000 (0:00:00.301) 0:00:01.595 ******** 2026-08-18 14:42:15.150349 | node1 | skipping: [node1] 2026-08-18 14:42:15.192327 | node1 | skipping: [node2] 2026-08-18 14:42:15.192452 | node1 | skipping: [node3] 2026-08-18 14:42:15.192629 | node1 | 2026-08-18 14:42:15.192834 | node1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-08-18 14:42:15.193033 | node1 | Tuesday 18 August 2026 14:42:15 +0000 (0:00:00.077) 0:00:01.672 ******** 2026-08-18 14:42:15.530561 | node1 | changed: [node2] 2026-08-18 14:42:15.530749 | node1 | changed: [node3] 2026-08-18 14:42:15.530964 | node1 | changed: [node1] 2026-08-18 14:42:15.531239 | node1 | 2026-08-18 14:42:15.531467 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:42:15.531684 | node1 | Tuesday 18 August 2026 14:42:15 +0000 (0:00:00.337) 0:00:02.010 ******** 2026-08-18 14:42:15.641960 | node1 | ok: [node2] => { 2026-08-18 14:42:15.642126 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:42:15.642344 | node1 | } 2026-08-18 14:42:15.642525 | node1 | ok: [node3] => { 2026-08-18 14:42:15.642764 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:42:15.642932 | node1 | } 2026-08-18 14:42:15.643167 | node1 | ok: [node1] => { 2026-08-18 14:42:15.643378 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:42:15.643551 | node1 | } 2026-08-18 14:42:15.643725 | node1 | 2026-08-18 14:42:15.643927 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:42:15.644123 | node1 | Tuesday 18 August 2026 14:42:15 +0000 (0:00:00.111) 0:00:02.122 ******** 2026-08-18 14:42:16.455904 | node1 | changed: [node3] 2026-08-18 14:42:16.456092 | node1 | changed: [node2] 2026-08-18 14:42:16.456300 | node1 | changed: [node1] 2026-08-18 14:42:16.456487 | node1 | 2026-08-18 14:42:16.456696 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:42:16.456917 | node1 | Tuesday 18 August 2026 14:42:16 +0000 (0:00:00.813) 0:00:02.935 ******** 2026-08-18 14:42:16.546544 | node1 | skipping: [node1] 2026-08-18 14:42:16.546722 | node1 | skipping: [node2] 2026-08-18 14:42:16.546914 | node1 | skipping: [node3] 2026-08-18 14:42:16.547092 | node1 | 2026-08-18 14:42:16.547299 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:42:16.547498 | node1 | Tuesday 18 August 2026 14:42:16 +0000 (0:00:00.090) 0:00:03.026 ******** 2026-08-18 14:42:16.642303 | node1 | skipping: [node1] 2026-08-18 14:42:16.642502 | node1 | skipping: [node2] 2026-08-18 14:42:16.642701 | node1 | skipping: [node3] 2026-08-18 14:42:16.642895 | node1 | 2026-08-18 14:42:16.643102 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:42:16.643302 | node1 | Tuesday 18 August 2026 14:42:16 +0000 (0:00:00.095) 0:00:03.122 ******** 2026-08-18 14:42:16.873051 | node1 | ok: [node1] 2026-08-18 14:42:16.873241 | node1 | ok: [node2] 2026-08-18 14:42:16.873499 | node1 | ok: [node3] 2026-08-18 14:42:16.873660 | node1 | 2026-08-18 14:42:16.873927 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:42:16.874164 | node1 | Tuesday 18 August 2026 14:42:16 +0000 (0:00:00.230) 0:00:03.352 ******** 2026-08-18 14:42:18.329204 | node1 | ok: [node2] 2026-08-18 14:42:18.329292 | node1 | ok: [node3] 2026-08-18 14:42:18.329303 | node1 | ok: [node1] 2026-08-18 14:42:18.329319 | node1 | 2026-08-18 14:42:18.329758 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:42:18.329826 | node1 | Tuesday 18 August 2026 14:42:18 +0000 (0:00:01.455) 0:00:04.808 ******** 2026-08-18 14:42:18.442236 | node1 | ok: [node1] => { 2026-08-18 14:42:18.442384 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:42:18.442594 | node1 | } 2026-08-18 14:42:18.442914 | node1 | ok: [node2] => { 2026-08-18 14:42:18.443194 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:42:18.443448 | node1 | } 2026-08-18 14:42:18.443705 | node1 | ok: [node3] => { 2026-08-18 14:42:18.443979 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:42:18.444226 | node1 | } 2026-08-18 14:42:18.444463 | node1 | 2026-08-18 14:42:18.444919 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:42:18.445386 | node1 | Tuesday 18 August 2026 14:42:18 +0000 (0:00:00.112) 0:00:04.921 ******** 2026-08-18 14:42:19.514936 | node1 | changed: [node3] 2026-08-18 14:42:19.515018 | node1 | changed: [node1] 2026-08-18 14:42:19.515029 | node1 | changed: [node2] 2026-08-18 14:42:19.515088 | node1 | 2026-08-18 14:42:19.515199 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:42:19.515371 | node1 | Tuesday 18 August 2026 14:42:19 +0000 (0:00:01.073) 0:00:05.994 ******** 2026-08-18 14:42:22.390171 | node1 | changed: [node3] 2026-08-18 14:42:22.390394 | node1 | changed: [node1] 2026-08-18 14:42:22.390408 | node1 | changed: [node2] 2026-08-18 14:42:22.390417 | node1 | 2026-08-18 14:42:22.390477 | node1 | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-08-18 14:42:22.390656 | node1 | Tuesday 18 August 2026 14:42:22 +0000 (0:00:02.875) 0:00:08.870 ******** 2026-08-18 14:42:22.426753 | node1 | skipping: [node1] 2026-08-18 14:42:22.455366 | node1 | skipping: [node2] 2026-08-18 14:42:22.455438 | node1 | skipping: [node3] 2026-08-18 14:42:22.455763 | node1 | 2026-08-18 14:42:22.455820 | node1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-08-18 14:42:22.455827 | node1 | Tuesday 18 August 2026 14:42:22 +0000 (0:00:00.065) 0:00:08.935 ******** 2026-08-18 14:42:22.490408 | node1 | skipping: [node1] 2026-08-18 14:42:22.525485 | node1 | skipping: [node2] 2026-08-18 14:42:22.525530 | node1 | skipping: [node3] 2026-08-18 14:42:22.525895 | node1 | 2026-08-18 14:42:22.525970 | node1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-08-18 14:42:22.525978 | node1 | Tuesday 18 August 2026 14:42:22 +0000 (0:00:00.069) 0:00:09.005 ******** 2026-08-18 14:42:22.567101 | node1 | skipping: [node1] 2026-08-18 14:42:22.597256 | node1 | skipping: [node2] 2026-08-18 14:42:22.597317 | node1 | skipping: [node3] 2026-08-18 14:42:22.597616 | node1 | 2026-08-18 14:42:22.597676 | node1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-08-18 14:42:22.597696 | node1 | Tuesday 18 August 2026 14:42:22 +0000 (0:00:00.071) 0:00:09.077 ******** 2026-08-18 14:42:27.859666 | node1 | changed: [node2] 2026-08-18 14:42:27.859756 | node1 | changed: [node1] 2026-08-18 14:42:27.859767 | node1 | changed: [node3] 2026-08-18 14:42:27.860049 | node1 | 2026-08-18 14:42:27.860091 | node1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-08-18 14:42:27.860097 | node1 | Tuesday 18 August 2026 14:42:27 +0000 (0:00:05.262) 0:00:14.339 ******** 2026-08-18 14:42:28.417349 | node1 | changed: [node3] 2026-08-18 14:42:28.418076 | node1 | changed: [node2] 2026-08-18 14:42:28.418222 | node1 | changed: [node1] 2026-08-18 14:42:28.418228 | node1 | 2026-08-18 14:42:28.418233 | node1 | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-08-18 14:42:28.418238 | node1 | Tuesday 18 August 2026 14:42:28 +0000 (0:00:00.557) 0:00:14.897 ******** 2026-08-18 14:42:29.340846 | node1 | changed: [node1] => (item={'path': '/etc/containerd'}) 2026-08-18 14:42:29.340902 | node1 | changed: [node2] => (item={'path': '/etc/containerd'}) 2026-08-18 14:42:29.340907 | node1 | changed: [node3] => (item={'path': '/etc/containerd'}) 2026-08-18 14:42:29.340912 | node1 | changed: [node2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:42:29.340918 | node1 | changed: [node3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:42:29.340922 | node1 | changed: [node1] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:42:29.340926 | node1 | changed: [node3] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:42:29.340930 | node1 | changed: [node2] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:42:29.340934 | node1 | changed: [node1] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:42:29.340939 | node1 | changed: [node3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:42:29.340943 | node1 | changed: [node2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:42:29.340947 | node1 | changed: [node1] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:42:29.340951 | node1 | changed: [node3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:42:29.340956 | node1 | changed: [node2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:42:29.340966 | node1 | changed: [node1] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:42:29.340970 | node1 | 2026-08-18 14:42:29.340975 | node1 | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-08-18 14:42:29.340979 | node1 | Tuesday 18 August 2026 14:42:29 +0000 (0:00:00.921) 0:00:15.819 ******** 2026-08-18 14:42:29.958659 | node1 | changed: [node1] 2026-08-18 14:42:29.958737 | node1 | changed: [node2] 2026-08-18 14:42:29.958770 | node1 | changed: [node3] 2026-08-18 14:42:29.959830 | node1 | 2026-08-18 14:42:29.959882 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:42:29.959888 | node1 | Tuesday 18 August 2026 14:42:29 +0000 (0:00:00.598) 0:00:16.417 ******** 2026-08-18 14:42:29.959892 | node1 | 2026-08-18 14:42:29.959896 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:42:29.959900 | node1 | Tuesday 18 August 2026 14:42:29 +0000 (0:00:00.007) 0:00:16.424 ******** 2026-08-18 14:42:29.959904 | node1 | 2026-08-18 14:42:29.959908 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:42:29.959912 | node1 | Tuesday 18 August 2026 14:42:29 +0000 (0:00:00.006) 0:00:16.431 ******** 2026-08-18 14:42:29.959916 | node1 | 2026-08-18 14:42:29.959920 | node1 | RUNNING HANDLER [vexxhost.containers.containerd : Reload systemd] ************** 2026-08-18 14:42:29.959924 | node1 | Tuesday 18 August 2026 14:42:29 +0000 (0:00:00.007) 0:00:16.438 ******** 2026-08-18 14:42:30.888541 | node1 | ok: [node3] 2026-08-18 14:42:30.888594 | node1 | ok: [node1] 2026-08-18 14:42:30.888602 | node1 | ok: [node2] 2026-08-18 14:42:30.888622 | node1 | 2026-08-18 14:42:30.888629 | node1 | RUNNING HANDLER [vexxhost.containers.containerd : Restart containerd] ********** 2026-08-18 14:42:30.888635 | node1 | Tuesday 18 August 2026 14:42:30 +0000 (0:00:00.929) 0:00:17.368 ******** 2026-08-18 14:42:31.337724 | node1 | changed: [node3] 2026-08-18 14:42:31.337812 | node1 | changed: [node2] 2026-08-18 14:42:31.338175 | node1 | changed: [node1] 2026-08-18 14:42:31.338231 | node1 | 2026-08-18 14:42:31.338237 | node1 | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-08-18 14:42:31.338243 | node1 | Tuesday 18 August 2026 14:42:31 +0000 (0:00:00.449) 0:00:17.817 ******** 2026-08-18 14:42:31.939759 | node1 | changed: [node3] 2026-08-18 14:42:31.939848 | node1 | changed: [node2] 2026-08-18 14:42:31.939862 | node1 | changed: [node1] 2026-08-18 14:42:31.940192 | node1 | 2026-08-18 14:42:31.940271 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:42:31.940278 | node1 | Tuesday 18 August 2026 14:42:31 +0000 (0:00:00.601) 0:00:18.419 ******** 2026-08-18 14:42:32.186924 | node1 | ok: [node1] 2026-08-18 14:42:32.187025 | node1 | ok: [node2] 2026-08-18 14:42:32.187036 | node1 | ok: [node3] 2026-08-18 14:42:32.187109 | node1 | 2026-08-18 14:42:32.187339 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:42:32.187409 | node1 | Tuesday 18 August 2026 14:42:32 +0000 (0:00:00.247) 0:00:18.667 ******** 2026-08-18 14:42:32.295514 | node1 | ok: [node1] => { 2026-08-18 14:42:32.296529 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:42:32.296592 | node1 | } 2026-08-18 14:42:32.296598 | node1 | ok: [node2] => { 2026-08-18 14:42:32.296604 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:42:32.296608 | node1 | } 2026-08-18 14:42:32.296612 | node1 | ok: [node3] => { 2026-08-18 14:42:32.296616 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:42:32.296620 | node1 | } 2026-08-18 14:42:32.296625 | node1 | 2026-08-18 14:42:32.296630 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:42:32.296636 | node1 | Tuesday 18 August 2026 14:42:32 +0000 (0:00:00.108) 0:00:18.775 ******** 2026-08-18 14:42:33.394376 | node1 | changed: [node2] 2026-08-18 14:42:33.394489 | node1 | changed: [node1] 2026-08-18 14:42:33.394501 | node1 | changed: [node3] 2026-08-18 14:42:33.394513 | node1 | 2026-08-18 14:42:33.394751 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:42:33.394816 | node1 | Tuesday 18 August 2026 14:42:33 +0000 (0:00:01.098) 0:00:19.874 ******** 2026-08-18 14:42:37.843163 | node1 | changed: [node3] 2026-08-18 14:42:37.843264 | node1 | changed: [node2] 2026-08-18 14:42:37.843280 | node1 | changed: [node1] 2026-08-18 14:42:37.843593 | node1 | 2026-08-18 14:42:37.843649 | node1 | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-08-18 14:42:37.843656 | node1 | Tuesday 18 August 2026 14:42:37 +0000 (0:00:04.448) 0:00:24.323 ******** 2026-08-18 14:42:39.097299 | node1 | ok: [node2] 2026-08-18 14:42:39.097384 | node1 | ok: [node3] 2026-08-18 14:42:39.097439 | node1 | ok: [node1] 2026-08-18 14:42:39.097735 | node1 | 2026-08-18 14:42:39.097804 | node1 | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-08-18 14:42:39.097810 | node1 | Tuesday 18 August 2026 14:42:39 +0000 (0:00:01.254) 0:00:25.577 ******** 2026-08-18 14:42:39.426640 | node1 | changed: [node3] 2026-08-18 14:42:39.426706 | node1 | changed: [node1] 2026-08-18 14:42:39.427127 | node1 | changed: [node2] 2026-08-18 14:42:39.427206 | node1 | 2026-08-18 14:42:39.427214 | node1 | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-08-18 14:42:39.427221 | node1 | Tuesday 18 August 2026 14:42:39 +0000 (0:00:00.327) 0:00:25.905 ******** 2026-08-18 14:42:39.921081 | node1 | changed: [node2] 2026-08-18 14:42:39.921201 | node1 | changed: [node1] 2026-08-18 14:42:39.921668 | node1 | changed: [node3] 2026-08-18 14:42:39.921750 | node1 | 2026-08-18 14:42:39.921760 | node1 | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-08-18 14:42:39.921768 | node1 | Tuesday 18 August 2026 14:42:39 +0000 (0:00:00.495) 0:00:26.400 ******** 2026-08-18 14:42:40.485480 | node1 | changed: [node2] => (item={'path': '/etc/docker'}) 2026-08-18 14:42:40.485561 | node1 | changed: [node1] => (item={'path': '/etc/docker'}) 2026-08-18 14:42:40.485617 | node1 | changed: [node3] => (item={'path': '/etc/docker'}) 2026-08-18 14:42:40.486460 | node1 | changed: [node2] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:42:40.486516 | node1 | changed: [node1] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:42:40.486521 | node1 | changed: [node3] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:42:40.486526 | node1 | changed: [node2] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:42:40.486530 | node1 | changed: [node3] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:42:40.486534 | node1 | changed: [node1] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:42:40.486539 | node1 | 2026-08-18 14:42:40.486545 | node1 | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-08-18 14:42:40.486550 | node1 | Tuesday 18 August 2026 14:42:40 +0000 (0:00:00.564) 0:00:26.965 ******** 2026-08-18 14:42:40.961018 | node1 | changed: [node2] 2026-08-18 14:42:40.961101 | node1 | changed: [node3] 2026-08-18 14:42:40.961154 | node1 | changed: [node1] 2026-08-18 14:42:40.961383 | node1 | 2026-08-18 14:42:40.961457 | node1 | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-08-18 14:42:40.961477 | node1 | Tuesday 18 August 2026 14:42:40 +0000 (0:00:00.475) 0:00:27.441 ******** 2026-08-18 14:42:41.457138 | node1 | changed: [node2] 2026-08-18 14:42:41.457229 | node1 | changed: [node1] 2026-08-18 14:42:41.457243 | node1 | changed: [node3] 2026-08-18 14:42:41.457383 | node1 | 2026-08-18 14:42:41.457480 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:42:41.457576 | node1 | Tuesday 18 August 2026 14:42:41 +0000 (0:00:00.475) 0:00:27.916 ******** 2026-08-18 14:42:41.457657 | node1 | 2026-08-18 14:42:41.457750 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:42:41.457849 | node1 | Tuesday 18 August 2026 14:42:41 +0000 (0:00:00.007) 0:00:27.923 ******** 2026-08-18 14:42:41.457925 | node1 | 2026-08-18 14:42:41.458017 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:42:41.458105 | node1 | Tuesday 18 August 2026 14:42:41 +0000 (0:00:00.007) 0:00:27.930 ******** 2026-08-18 14:42:41.458187 | node1 | 2026-08-18 14:42:41.458280 | node1 | RUNNING HANDLER [vexxhost.containers.containerd : Reload systemd] ************** 2026-08-18 14:42:41.458370 | node1 | Tuesday 18 August 2026 14:42:41 +0000 (0:00:00.006) 0:00:27.937 ******** 2026-08-18 14:42:42.161340 | node1 | ok: [node3] 2026-08-18 14:42:42.161533 | node1 | ok: [node2] 2026-08-18 14:42:42.161742 | node1 | ok: [node1] 2026-08-18 14:42:42.161943 | node1 | 2026-08-18 14:42:42.162173 | node1 | RUNNING HANDLER [vexxhost.containers.docker : Restart docker] ****************** 2026-08-18 14:42:42.162392 | node1 | Tuesday 18 August 2026 14:42:42 +0000 (0:00:00.703) 0:00:28.641 ******** 2026-08-18 14:42:43.367663 | node1 | changed: [node2] 2026-08-18 14:42:43.367744 | node1 | changed: [node1] 2026-08-18 14:42:43.368061 | node1 | changed: [node3] 2026-08-18 14:42:43.368077 | node1 | 2026-08-18 14:42:43.368083 | node1 | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-08-18 14:42:43.368088 | node1 | Tuesday 18 August 2026 14:42:43 +0000 (0:00:01.206) 0:00:29.848 ******** 2026-08-18 14:42:43.963376 | node1 | changed: [node1] 2026-08-18 14:42:43.963432 | node1 | changed: [node2] 2026-08-18 14:42:43.963437 | node1 | changed: [node3] 2026-08-18 14:42:43.963441 | node1 | 2026-08-18 14:42:43.963446 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:42:43.963451 | node1 | Tuesday 18 August 2026 14:42:43 +0000 (0:00:00.595) 0:00:30.443 ******** 2026-08-18 14:42:44.020809 | node1 | ok: [node1] => { 2026-08-18 14:42:44.020869 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:42:44.020875 | node1 | } 2026-08-18 14:42:44.020879 | node1 | ok: [node2] => { 2026-08-18 14:42:44.020883 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:42:44.020902 | node1 | } 2026-08-18 14:42:44.020906 | node1 | ok: [node3] => { 2026-08-18 14:42:44.020910 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:42:44.020913 | node1 | } 2026-08-18 14:42:44.020917 | node1 | 2026-08-18 14:42:44.020922 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:42:44.020926 | node1 | Tuesday 18 August 2026 14:42:44 +0000 (0:00:00.056) 0:00:30.500 ******** 2026-08-18 14:42:45.272829 | node1 | changed: [node1] 2026-08-18 14:42:45.272913 | node1 | changed: [node2] 2026-08-18 14:42:45.272924 | node1 | changed: [node3] 2026-08-18 14:42:45.273284 | node1 | 2026-08-18 14:42:45.273342 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:42:45.273349 | node1 | Tuesday 18 August 2026 14:42:45 +0000 (0:00:01.252) 0:00:31.753 ******** 2026-08-18 14:42:45.303951 | node1 | skipping: [node1] 2026-08-18 14:42:45.356379 | node1 | skipping: [node2] 2026-08-18 14:42:45.356444 | node1 | skipping: [node3] 2026-08-18 14:42:45.356568 | node1 | 2026-08-18 14:42:45.356796 | node1 | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-08-18 14:42:45.356835 | node1 | Tuesday 18 August 2026 14:42:45 +0000 (0:00:00.083) 0:00:31.836 ******** 2026-08-18 14:42:45.399824 | node1 | ok: [node1] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/ubuntu-22.04.yml) 2026-08-18 14:42:45.448873 | node1 | ok: [node2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/ubuntu-22.04.yml) 2026-08-18 14:42:45.448951 | node1 | ok: [node3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/ubuntu-22.04.yml) 2026-08-18 14:42:45.449293 | node1 | 2026-08-18 14:42:45.449351 | node1 | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-08-18 14:42:45.449357 | node1 | Tuesday 18 August 2026 14:42:45 +0000 (0:00:00.092) 0:00:31.929 ******** 2026-08-18 14:42:50.760976 | node1 | changed: [node2] 2026-08-18 14:42:50.761022 | node1 | changed: [node3] 2026-08-18 14:42:50.761027 | node1 | changed: [node1] 2026-08-18 14:42:50.761032 | node1 | 2026-08-18 14:42:50.761037 | node1 | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-08-18 14:42:50.761041 | node1 | Tuesday 18 August 2026 14:42:50 +0000 (0:00:05.311) 0:00:37.240 ******** 2026-08-18 14:42:51.416447 | node1 | ok: [node1] => (item=chronyd) 2026-08-18 14:42:51.416532 | node1 | ok: [node3] => (item=chronyd) 2026-08-18 14:42:51.416583 | node1 | ok: [node2] => (item=chronyd) 2026-08-18 14:42:51.417211 | node1 | ok: [node3] => (item=sshd) 2026-08-18 14:42:51.417472 | node1 | ok: [node2] => (item=sshd) 2026-08-18 14:42:51.417478 | node1 | ok: [node1] => (item=sshd) 2026-08-18 14:42:51.417483 | node1 | 2026-08-18 14:42:51.417487 | node1 | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-08-18 14:42:51.417493 | node1 | Tuesday 18 August 2026 14:42:51 +0000 (0:00:00.655) 0:00:37.896 ******** 2026-08-18 14:42:51.632065 | node1 | ok: [node1] 2026-08-18 14:42:51.632155 | node1 | ok: [node2] 2026-08-18 14:42:51.632167 | node1 | ok: [node3] 2026-08-18 14:42:51.632491 | node1 | 2026-08-18 14:42:51.632562 | node1 | TASK [vexxhost.ceph.cephadm : Ensure the cephadm SSH user exists] ************** 2026-08-18 14:42:51.632569 | node1 | Tuesday 18 August 2026 14:42:51 +0000 (0:00:00.216) 0:00:38.112 ******** 2026-08-18 14:42:52.058737 | node1 | changed: [node3] 2026-08-18 14:42:52.058816 | node1 | changed: [node1] 2026-08-18 14:42:52.058875 | node1 | changed: [node2] 2026-08-18 14:42:52.059222 | node1 | 2026-08-18 14:42:52.059299 | node1 | TASK [vexxhost.ceph.cephadm : Grant passwordless sudo to the cephadm SSH user] *** 2026-08-18 14:42:52.059305 | node1 | Tuesday 18 August 2026 14:42:52 +0000 (0:00:00.426) 0:00:38.538 ******** 2026-08-18 14:42:52.539948 | node1 | changed: [node2] 2026-08-18 14:42:52.540065 | node1 | changed: [node1] 2026-08-18 14:42:52.540077 | node1 | changed: [node3] 2026-08-18 14:42:52.540089 | node1 | 2026-08-18 14:42:52.540332 | node1 | TASK [vexxhost.ceph.cephadm : Log in to the container registry] **************** 2026-08-18 14:42:52.540394 | node1 | Tuesday 18 August 2026 14:42:52 +0000 (0:00:00.481) 0:00:39.020 ******** 2026-08-18 14:42:52.576879 | node1 | skipping: [node1] 2026-08-18 14:42:52.614613 | node1 | skipping: [node2] 2026-08-18 14:42:52.614704 | node1 | skipping: [node3] 2026-08-18 14:42:52.614715 | node1 | 2026-08-18 14:42:52.614775 | node1 | TASK [vexxhost.ceph.cephadm : Pre-pull the Ceph image] ************************* 2026-08-18 14:42:52.614930 | node1 | Tuesday 18 August 2026 14:42:52 +0000 (0:00:00.073) 0:00:39.093 ******** 2026-08-18 14:43:34.470609 | node1 | changed: [node3] 2026-08-18 14:43:34.470690 | node1 | changed: [node2] 2026-08-18 14:43:34.470704 | node1 | changed: [node1] 2026-08-18 14:43:34.471025 | node1 | 2026-08-18 14:43:34.471079 | node1 | TASK [vexxhost.ceph.cephadm : Pin CEPHADM_IMAGE for interactive login shells] *** 2026-08-18 14:43:34.471086 | node1 | Tuesday 18 August 2026 14:43:34 +0000 (0:00:41.857) 0:01:20.950 ******** 2026-08-18 14:43:34.979755 | node1 | changed: [node1] 2026-08-18 14:43:34.979843 | node1 | changed: [node2] 2026-08-18 14:43:34.979879 | node1 | changed: [node3] 2026-08-18 14:43:34.980134 | node1 | 2026-08-18 14:43:34.980240 | node1 | TASK [vexxhost.ceph.mon : Find this monitor's addresses on the public network] *** 2026-08-18 14:43:34.980268 | node1 | Tuesday 18 August 2026 14:43:34 +0000 (0:00:00.509) 0:01:21.459 ******** 2026-08-18 14:43:35.097882 | node1 | ok: [node2] 2026-08-18 14:43:35.097968 | node1 | ok: [node1] 2026-08-18 14:43:35.098519 | node1 | ok: [node3] 2026-08-18 14:43:35.098570 | node1 | 2026-08-18 14:43:35.098577 | node1 | TASK [vexxhost.ceph.mon : Assert exactly one address matches the public network] *** 2026-08-18 14:43:35.098584 | node1 | Tuesday 18 August 2026 14:43:35 +0000 (0:00:00.117) 0:01:21.577 ******** 2026-08-18 14:43:35.176907 | node1 | ok: [node1] => { 2026-08-18 14:43:35.176979 | node1 | "changed": false, 2026-08-18 14:43:35.177042 | node1 | "msg": "All assertions passed" 2026-08-18 14:43:35.178161 | node1 | } 2026-08-18 14:43:35.178215 | node1 | ok: [node2] => { 2026-08-18 14:43:35.178221 | node1 | "changed": false, 2026-08-18 14:43:35.178226 | node1 | "msg": "All assertions passed" 2026-08-18 14:43:35.178232 | node1 | } 2026-08-18 14:43:35.178236 | node1 | ok: [node3] => { 2026-08-18 14:43:35.178240 | node1 | "changed": false, 2026-08-18 14:43:35.178244 | node1 | "msg": "All assertions passed" 2026-08-18 14:43:35.178248 | node1 | } 2026-08-18 14:43:35.178252 | node1 | 2026-08-18 14:43:35.178256 | node1 | TASK [vexxhost.ceph.mon : Set this monitor's Ceph public IP] ******************* 2026-08-18 14:43:35.178262 | node1 | Tuesday 18 August 2026 14:43:35 +0000 (0:00:00.079) 0:01:21.657 ******** 2026-08-18 14:43:35.258118 | node1 | ok: [node1] 2026-08-18 14:43:35.258144 | node1 | ok: [node2] 2026-08-18 14:43:35.258151 | node1 | ok: [node3] 2026-08-18 14:43:35.258157 | node1 | 2026-08-18 14:43:35.258162 | node1 | TASK [vexxhost.ceph.mon : Check whether any monitor is already bootstrapped] *** 2026-08-18 14:43:35.258168 | node1 | Tuesday 18 August 2026 14:43:35 +0000 (0:00:00.080) 0:01:21.737 ******** 2026-08-18 14:43:35.804141 | node1 | ok: [node1] => (item=node1) 2026-08-18 14:43:35.804227 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 14:43:35.804284 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 14:43:35.804658 | node1 | 2026-08-18 14:43:35.804721 | node1 | TASK [vexxhost.ceph.mon : Select the bootstrap node (existing one, else the first monitor)] *** 2026-08-18 14:43:35.804728 | node1 | Tuesday 18 August 2026 14:43:35 +0000 (0:00:00.546) 0:01:22.284 ******** 2026-08-18 14:43:35.886655 | node1 | ok: [node1] 2026-08-18 14:43:35.886701 | node1 | ok: [node2] 2026-08-18 14:43:35.887141 | node1 | ok: [node3] 2026-08-18 14:43:35.887192 | node1 | 2026-08-18 14:43:35.887198 | node1 | TASK [vexxhost.ceph.mon : Bootstrap the cluster] ******************************* 2026-08-18 14:43:35.887204 | node1 | Tuesday 18 August 2026 14:43:35 +0000 (0:00:00.082) 0:01:22.366 ******** 2026-08-18 14:43:36.017646 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/mon/tasks/bootstrap.yml for node1, node2, node3 2026-08-18 14:43:36.017705 | node1 | 2026-08-18 14:43:36.017946 | node1 | TASK [vexxhost.ceph.mon : Ensure /etc/ceph exists] ***************************** 2026-08-18 14:43:36.017999 | node1 | Tuesday 18 August 2026 14:43:36 +0000 (0:00:00.131) 0:01:22.497 ******** 2026-08-18 14:43:36.228855 | node1 | skipping: [node2] 2026-08-18 14:43:36.228924 | node1 | skipping: [node3] 2026-08-18 14:43:36.229011 | node1 | changed: [node1] 2026-08-18 14:43:36.229249 | node1 | 2026-08-18 14:43:36.229370 | node1 | TASK [vexxhost.ceph.mon : Write the caller-provided SSH private key] *********** 2026-08-18 14:43:36.229384 | node1 | Tuesday 18 August 2026 14:43:36 +0000 (0:00:00.210) 0:01:22.708 ******** 2026-08-18 14:43:36.296741 | node1 | skipping: [node1] 2026-08-18 14:43:36.296820 | node1 | skipping: [node2] 2026-08-18 14:43:36.296834 | node1 | skipping: [node3] 2026-08-18 14:43:36.297208 | node1 | 2026-08-18 14:43:36.297293 | node1 | TASK [vexxhost.ceph.mon : Write the caller-provided SSH public key] ************ 2026-08-18 14:43:36.297307 | node1 | Tuesday 18 August 2026 14:43:36 +0000 (0:00:00.068) 0:01:22.776 ******** 2026-08-18 14:43:36.339497 | node1 | skipping: [node1] 2026-08-18 14:43:36.373791 | node1 | skipping: [node2] 2026-08-18 14:43:36.373887 | node1 | skipping: [node3] 2026-08-18 14:43:36.373900 | node1 | 2026-08-18 14:43:36.374008 | node1 | TASK [vexxhost.ceph.mon : Generate a cephadm SSH keypair when none was provided] *** 2026-08-18 14:43:36.374144 | node1 | Tuesday 18 August 2026 14:43:36 +0000 (0:00:00.077) 0:01:22.854 ******** 2026-08-18 14:43:38.551488 | node1 | skipping: [node2] 2026-08-18 14:43:38.551571 | node1 | skipping: [node3] 2026-08-18 14:43:38.551585 | node1 | changed: [node1] 2026-08-18 14:43:38.551914 | node1 | 2026-08-18 14:43:38.551976 | node1 | TASK [vexxhost.ceph.mon : Create a temporary ceph.conf for bootstrap overrides] *** 2026-08-18 14:43:38.551983 | node1 | Tuesday 18 August 2026 14:43:38 +0000 (0:00:02.177) 0:01:25.031 ******** 2026-08-18 14:43:38.858641 | node1 | skipping: [node2] 2026-08-18 14:43:38.858725 | node1 | skipping: [node3] 2026-08-18 14:43:38.859136 | node1 | changed: [node1] 2026-08-18 14:43:38.859205 | node1 | 2026-08-18 14:43:38.859211 | node1 | TASK [vexxhost.ceph.mon : Populate bootstrap ceph.conf overrides] ************** 2026-08-18 14:43:38.859217 | node1 | Tuesday 18 August 2026 14:43:38 +0000 (0:00:00.305) 0:01:25.337 ******** 2026-08-18 14:43:38.898562 | node1 | skipping: [node1] 2026-08-18 14:43:38.924598 | node1 | skipping: [node2] 2026-08-18 14:43:38.924640 | node1 | skipping: [node3] 2026-08-18 14:43:38.924753 | node1 | 2026-08-18 14:43:38.925010 | node1 | TASK [vexxhost.ceph.mon : Bootstrap the cluster] ******************************* 2026-08-18 14:43:38.925072 | node1 | Tuesday 18 August 2026 14:43:38 +0000 (0:00:00.067) 0:01:25.405 ******** 2026-08-18 14:44:20.642413 | node1 | skipping: [node2] 2026-08-18 14:44:20.642513 | node1 | skipping: [node3] 2026-08-18 14:44:20.642531 | node1 | changed: [node1] 2026-08-18 14:44:20.642731 | node1 | 2026-08-18 14:44:20.642825 | node1 | TASK [vexxhost.ceph.mon : Remove the temporary ceph.conf] ********************** 2026-08-18 14:44:20.642839 | node1 | Tuesday 18 August 2026 14:44:20 +0000 (0:00:41.717) 0:02:07.122 ******** 2026-08-18 14:44:20.852828 | node1 | skipping: [node2] 2026-08-18 14:44:20.852890 | node1 | skipping: [node3] 2026-08-18 14:44:20.853258 | node1 | changed: [node1] 2026-08-18 14:44:20.853344 | node1 | 2026-08-18 14:44:20.853356 | node1 | TASK [Register this monitor host] ********************************************** 2026-08-18 14:44:20.853377 | node1 | Tuesday 18 August 2026 14:44:20 +0000 (0:00:00.210) 0:02:07.332 ******** 2026-08-18 14:44:20.990375 | node1 | included: vexxhost.ceph.cephadm_host for node1, node2, node3 2026-08-18 14:44:20.990458 | node1 | 2026-08-18 14:44:20.990470 | node1 | TASK [vexxhost.ceph.cephadm_host : Get the cephadm public key] ***************** 2026-08-18 14:44:20.990625 | node1 | Tuesday 18 August 2026 14:44:20 +0000 (0:00:00.137) 0:02:07.470 ******** 2026-08-18 14:44:22.532363 | node1 | ok: [node1] 2026-08-18 14:44:22.532451 | node1 | 2026-08-18 14:44:22.532458 | node1 | TASK [vexxhost.ceph.cephadm_host : Resolve the cluster SSH public key to authorise] *** 2026-08-18 14:44:22.532465 | node1 | Tuesday 18 August 2026 14:44:22 +0000 (0:00:01.540) 0:02:09.011 ******** 2026-08-18 14:44:22.612802 | node1 | ok: [node1] 2026-08-18 14:44:22.612861 | node1 | ok: [node2] 2026-08-18 14:44:22.612866 | node1 | ok: [node3] 2026-08-18 14:44:22.612871 | node1 | 2026-08-18 14:44:22.612876 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert a cluster SSH key was resolved] ****** 2026-08-18 14:44:22.612881 | node1 | Tuesday 18 August 2026 14:44:22 +0000 (0:00:00.081) 0:02:09.092 ******** 2026-08-18 14:44:22.690975 | node1 | ok: [node1] => { 2026-08-18 14:44:22.691054 | node1 | "changed": false, 2026-08-18 14:44:22.691060 | node1 | "msg": "All assertions passed" 2026-08-18 14:44:22.691066 | node1 | } 2026-08-18 14:44:22.691070 | node1 | ok: [node2] => { 2026-08-18 14:44:22.691074 | node1 | "changed": false, 2026-08-18 14:44:22.691078 | node1 | "msg": "All assertions passed" 2026-08-18 14:44:22.691082 | node1 | } 2026-08-18 14:44:22.691085 | node1 | ok: [node3] => { 2026-08-18 14:44:22.691089 | node1 | "changed": false, 2026-08-18 14:44:22.691093 | node1 | "msg": "All assertions passed" 2026-08-18 14:44:22.691097 | node1 | } 2026-08-18 14:44:22.691101 | node1 | 2026-08-18 14:44:22.691106 | node1 | TASK [vexxhost.ceph.cephadm_host : Authorise the cluster SSH key for the cephadm user] *** 2026-08-18 14:44:22.691111 | node1 | Tuesday 18 August 2026 14:44:22 +0000 (0:00:00.077) 0:02:09.170 ******** 2026-08-18 14:44:23.137514 | node1 | changed: [node3] 2026-08-18 14:44:23.137613 | node1 | ok: [node1] 2026-08-18 14:44:23.138167 | node1 | changed: [node2] 2026-08-18 14:44:23.138246 | node1 | 2026-08-18 14:44:23.138256 | node1 | TASK [vexxhost.ceph.cephadm_host : Derive this host's cephadm labels from inventory membership] *** 2026-08-18 14:44:23.138266 | node1 | Tuesday 18 August 2026 14:44:23 +0000 (0:00:00.447) 0:02:09.617 ******** 2026-08-18 14:44:23.232615 | node1 | ok: [node1] 2026-08-18 14:44:23.232675 | node1 | ok: [node2] 2026-08-18 14:44:23.232684 | node1 | ok: [node3] 2026-08-18 14:44:23.232693 | node1 | 2026-08-18 14:44:23.232700 | node1 | TASK [vexxhost.ceph.cephadm_host : Find this host's addresses on the public network] *** 2026-08-18 14:44:23.232706 | node1 | Tuesday 18 August 2026 14:44:23 +0000 (0:00:00.094) 0:02:09.711 ******** 2026-08-18 14:44:23.334002 | node1 | ok: [node2] 2026-08-18 14:44:23.334054 | node1 | ok: [node1] 2026-08-18 14:44:23.334059 | node1 | ok: [node3] 2026-08-18 14:44:23.334063 | node1 | 2026-08-18 14:44:23.334075 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert exactly one address matches the public network] *** 2026-08-18 14:44:23.334080 | node1 | Tuesday 18 August 2026 14:44:23 +0000 (0:00:00.102) 0:02:09.813 ******** 2026-08-18 14:44:23.401741 | node1 | ok: [node1] => { 2026-08-18 14:44:23.401791 | node1 | "changed": false, 2026-08-18 14:44:23.401797 | node1 | "msg": "All assertions passed" 2026-08-18 14:44:23.401802 | node1 | } 2026-08-18 14:44:23.401806 | node1 | ok: [node2] => { 2026-08-18 14:44:23.401810 | node1 | "changed": false, 2026-08-18 14:44:23.401814 | node1 | "msg": "All assertions passed" 2026-08-18 14:44:23.401818 | node1 | } 2026-08-18 14:44:23.401821 | node1 | ok: [node3] => { 2026-08-18 14:44:23.401825 | node1 | "changed": false, 2026-08-18 14:44:23.401829 | node1 | "msg": "All assertions passed" 2026-08-18 14:44:23.401833 | node1 | } 2026-08-18 14:44:23.401837 | node1 | 2026-08-18 14:44:23.401841 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host (labels)] ***************** 2026-08-18 14:44:23.401846 | node1 | Tuesday 18 August 2026 14:44:23 +0000 (0:00:00.067) 0:02:09.880 ******** 2026-08-18 14:44:50.102455 | node1 | changed: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:44:50.102520 | node1 | changed: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:44:50.102526 | node1 | changed: [node1] 2026-08-18 14:44:50.102531 | node1 | 2026-08-18 14:44:50.102536 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host with a CRUSH location] **** 2026-08-18 14:44:50.102542 | node1 | Tuesday 18 August 2026 14:44:50 +0000 (0:00:26.701) 0:02:36.581 ******** 2026-08-18 14:44:50.199876 | node1 | skipping: [node1] 2026-08-18 14:44:50.199947 | node1 | skipping: [node2] 2026-08-18 14:44:50.199952 | node1 | skipping: [node3] 2026-08-18 14:44:50.199957 | node1 | 2026-08-18 14:44:50.199962 | node1 | TASK [vexxhost.ceph.mon : Apply the monitor service spec] ********************** 2026-08-18 14:44:50.199967 | node1 | Tuesday 18 August 2026 14:44:50 +0000 (0:00:00.097) 0:02:36.679 ******** 2026-08-18 14:44:53.080972 | node1 | changed: [node1] 2026-08-18 14:44:53.081084 | node1 | 2026-08-18 14:44:53.081097 | node1 | TASK [vexxhost.ceph.mon : Wait for this monitor's daemon to be present] ******** 2026-08-18 14:44:53.081266 | node1 | Tuesday 18 August 2026 14:44:53 +0000 (0:00:02.881) 0:02:39.561 ******** 2026-08-18 14:44:54.833914 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:44:54.834027 | node1 | ok: [node1] 2026-08-18 14:44:54.834039 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:44:54.834132 | node1 | 2026-08-18 14:44:54.834403 | node1 | TASK [vexxhost.ceph.mgr : Locate the cluster admin host] *********************** 2026-08-18 14:44:54.834488 | node1 | Tuesday 18 August 2026 14:44:54 +0000 (0:00:01.752) 0:02:41.313 ******** 2026-08-18 14:44:55.409305 | node1 | ok: [node1] => (item=node1) 2026-08-18 14:44:55.409412 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 14:44:55.409427 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 14:44:55.409510 | node1 | 2026-08-18 14:44:55.409609 | node1 | TASK [vexxhost.ceph.mgr : Select the admin host] ******************************* 2026-08-18 14:44:55.409712 | node1 | Tuesday 18 August 2026 14:44:55 +0000 (0:00:00.575) 0:02:41.889 ******** 2026-08-18 14:44:55.501272 | node1 | ok: [node1] 2026-08-18 14:44:55.501439 | node1 | ok: [node2] 2026-08-18 14:44:55.501636 | node1 | ok: [node3] 2026-08-18 14:44:55.501816 | node1 | 2026-08-18 14:44:55.502049 | node1 | TASK [Register this manager host] ********************************************** 2026-08-18 14:44:55.502252 | node1 | Tuesday 18 August 2026 14:44:55 +0000 (0:00:00.092) 0:02:41.981 ******** 2026-08-18 14:44:55.635509 | node1 | included: vexxhost.ceph.cephadm_host for node1, node2, node3 2026-08-18 14:44:55.635618 | node1 | 2026-08-18 14:44:55.635831 | node1 | TASK [vexxhost.ceph.cephadm_host : Get the cephadm public key] ***************** 2026-08-18 14:44:55.636026 | node1 | Tuesday 18 August 2026 14:44:55 +0000 (0:00:00.133) 0:02:42.115 ******** 2026-08-18 14:45:03.185395 | node1 | ok: [node1] 2026-08-18 14:45:03.185518 | node1 | 2026-08-18 14:45:03.185674 | node1 | TASK [vexxhost.ceph.cephadm_host : Resolve the cluster SSH public key to authorise] *** 2026-08-18 14:45:03.185803 | node1 | Tuesday 18 August 2026 14:45:03 +0000 (0:00:07.549) 0:02:49.665 ******** 2026-08-18 14:45:03.257279 | node1 | ok: [node1] 2026-08-18 14:45:03.257491 | node1 | ok: [node2] 2026-08-18 14:45:03.257694 | node1 | ok: [node3] 2026-08-18 14:45:03.257879 | node1 | 2026-08-18 14:45:03.258158 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert a cluster SSH key was resolved] ****** 2026-08-18 14:45:03.258362 | node1 | Tuesday 18 August 2026 14:45:03 +0000 (0:00:00.071) 0:02:49.736 ******** 2026-08-18 14:45:03.322608 | node1 | ok: [node1] => { 2026-08-18 14:45:03.322813 | node1 | "changed": false, 2026-08-18 14:45:03.323014 | node1 | "msg": "All assertions passed" 2026-08-18 14:45:03.323206 | node1 | } 2026-08-18 14:45:03.323403 | node1 | ok: [node2] => { 2026-08-18 14:45:03.323589 | node1 | "changed": false, 2026-08-18 14:45:03.323792 | node1 | "msg": "All assertions passed" 2026-08-18 14:45:03.323971 | node1 | } 2026-08-18 14:45:03.324164 | node1 | ok: [node3] => { 2026-08-18 14:45:03.324355 | node1 | "changed": false, 2026-08-18 14:45:03.324546 | node1 | "msg": "All assertions passed" 2026-08-18 14:45:03.324728 | node1 | } 2026-08-18 14:45:03.324904 | node1 | 2026-08-18 14:45:03.325154 | node1 | TASK [vexxhost.ceph.cephadm_host : Authorise the cluster SSH key for the cephadm user] *** 2026-08-18 14:45:03.325429 | node1 | Tuesday 18 August 2026 14:45:03 +0000 (0:00:00.065) 0:02:49.802 ******** 2026-08-18 14:45:03.591533 | node1 | ok: [node1] 2026-08-18 14:45:03.591621 | node1 | ok: [node2] 2026-08-18 14:45:03.591743 | node1 | ok: [node3] 2026-08-18 14:45:03.591865 | node1 | 2026-08-18 14:45:03.592037 | node1 | TASK [vexxhost.ceph.cephadm_host : Derive this host's cephadm labels from inventory membership] *** 2026-08-18 14:45:03.592191 | node1 | Tuesday 18 August 2026 14:45:03 +0000 (0:00:00.269) 0:02:50.071 ******** 2026-08-18 14:45:03.660347 | node1 | ok: [node1] 2026-08-18 14:45:03.660477 | node1 | ok: [node2] 2026-08-18 14:45:03.660578 | node1 | ok: [node3] 2026-08-18 14:45:03.660694 | node1 | 2026-08-18 14:45:03.660835 | node1 | TASK [vexxhost.ceph.cephadm_host : Find this host's addresses on the public network] *** 2026-08-18 14:45:03.660963 | node1 | Tuesday 18 August 2026 14:45:03 +0000 (0:00:00.068) 0:02:50.140 ******** 2026-08-18 14:45:03.763866 | node1 | ok: [node1] 2026-08-18 14:45:03.764124 | node1 | ok: [node2] 2026-08-18 14:45:03.764292 | node1 | ok: [node3] 2026-08-18 14:45:03.764483 | node1 | 2026-08-18 14:45:03.764709 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert exactly one address matches the public network] *** 2026-08-18 14:45:03.764988 | node1 | Tuesday 18 August 2026 14:45:03 +0000 (0:00:00.103) 0:02:50.243 ******** 2026-08-18 14:45:03.842251 | node1 | ok: [node1] => { 2026-08-18 14:45:03.842336 | node1 | "changed": false, 2026-08-18 14:45:03.842454 | node1 | "msg": "All assertions passed" 2026-08-18 14:45:03.842547 | node1 | } 2026-08-18 14:45:03.842676 | node1 | ok: [node2] => { 2026-08-18 14:45:03.842774 | node1 | "changed": false, 2026-08-18 14:45:03.842895 | node1 | "msg": "All assertions passed" 2026-08-18 14:45:03.843009 | node1 | } 2026-08-18 14:45:03.843130 | node1 | ok: [node3] => { 2026-08-18 14:45:03.843249 | node1 | "changed": false, 2026-08-18 14:45:03.843381 | node1 | "msg": "All assertions passed" 2026-08-18 14:45:03.843482 | node1 | } 2026-08-18 14:45:03.843590 | node1 | 2026-08-18 14:45:03.843694 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host (labels)] ***************** 2026-08-18 14:45:03.843810 | node1 | Tuesday 18 August 2026 14:45:03 +0000 (0:00:00.078) 0:02:50.322 ******** 2026-08-18 14:45:05.411429 | node1 | ok: [node1] 2026-08-18 14:45:05.411519 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:45:05.411532 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:45:05.411542 | node1 | 2026-08-18 14:45:05.411553 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host with a CRUSH location] **** 2026-08-18 14:45:05.411563 | node1 | Tuesday 18 August 2026 14:45:05 +0000 (0:00:01.567) 0:02:51.890 ******** 2026-08-18 14:45:05.511053 | node1 | skipping: [node1] 2026-08-18 14:45:05.511110 | node1 | skipping: [node2] 2026-08-18 14:45:05.511115 | node1 | skipping: [node3] 2026-08-18 14:45:05.511119 | node1 | 2026-08-18 14:45:05.511124 | node1 | TASK [vexxhost.ceph.mgr : Apply the manager service spec] ********************** 2026-08-18 14:45:05.511130 | node1 | Tuesday 18 August 2026 14:45:05 +0000 (0:00:00.100) 0:02:51.990 ******** 2026-08-18 14:45:08.371284 | node1 | changed: [node1] 2026-08-18 14:45:08.371357 | node1 | 2026-08-18 14:45:08.371630 | node1 | TASK [vexxhost.ceph.mgr : Wait for this manager's daemon to be present] ******** 2026-08-18 14:45:08.371651 | node1 | Tuesday 18 August 2026 14:45:08 +0000 (0:00:02.860) 0:02:54.851 ******** 2026-08-18 14:45:09.964963 | node1 | ok: [node1] 2026-08-18 14:45:09.965038 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:45:09.965459 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:45:09.965513 | node1 | 2026-08-18 14:45:09.965519 | node1 | TASK [vexxhost.ceph.mgr : Configure ceph-mgr modules] ************************** 2026-08-18 14:45:09.965524 | node1 | Tuesday 18 August 2026 14:45:09 +0000 (0:00:01.593) 0:02:56.445 ******** 2026-08-18 14:45:19.698147 | node1 | changed: [node1] => (item={'name': 'diskprediction_local'}) 2026-08-18 14:45:19.698233 | node1 | changed: [node1] => (item={'name': 'dashboard'}) 2026-08-18 14:45:19.698274 | node1 | ok: [node1] => (item={'name': 'iostat'}) 2026-08-18 14:45:19.698376 | node1 | changed: [node1] => (item={'name': 'prometheus'}) 2026-08-18 14:45:19.698466 | node1 | changed: [node1] => (item={'name': 'rgw'}) 2026-08-18 14:45:19.698551 | node1 | 2026-08-18 14:45:19.698645 | node1 | PLAY [Deploy Ceph OSDs] ******************************************************** 2026-08-18 14:45:19.698721 | node1 | 2026-08-18 14:45:19.698816 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:45:19.698909 | node1 | Tuesday 18 August 2026 14:45:19 +0000 (0:00:09.733) 0:03:06.178 ******** 2026-08-18 14:45:20.620544 | node1 | ok: [node2] 2026-08-18 14:45:20.620612 | node1 | ok: [node1] 2026-08-18 14:45:20.620684 | node1 | ok: [node3] 2026-08-18 14:45:20.620770 | node1 | 2026-08-18 14:45:20.620876 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:45:20.620991 | node1 | Tuesday 18 August 2026 14:45:20 +0000 (0:00:00.922) 0:03:07.100 ******** 2026-08-18 14:45:20.847506 | node1 | ok: [node2] 2026-08-18 14:45:20.847576 | node1 | ok: [node1] 2026-08-18 14:45:20.847635 | node1 | ok: [node3] 2026-08-18 14:45:20.847720 | node1 | 2026-08-18 14:45:20.847818 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:45:20.847910 | node1 | Tuesday 18 August 2026 14:45:20 +0000 (0:00:00.227) 0:03:07.327 ******** 2026-08-18 14:45:20.919500 | node1 | skipping: [node1] 2026-08-18 14:45:20.919994 | node1 | skipping: [node2] 2026-08-18 14:45:20.920047 | node1 | skipping: [node3] 2026-08-18 14:45:20.920066 | node1 | 2026-08-18 14:45:20.920071 | node1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-08-18 14:45:20.920076 | node1 | Tuesday 18 August 2026 14:45:20 +0000 (0:00:00.072) 0:03:07.399 ******** 2026-08-18 14:45:21.149168 | node1 | ok: [node1] 2026-08-18 14:45:21.149217 | node1 | ok: [node2] 2026-08-18 14:45:21.149222 | node1 | ok: [node3] 2026-08-18 14:45:21.149227 | node1 | 2026-08-18 14:45:21.149232 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:45:21.149237 | node1 | Tuesday 18 August 2026 14:45:21 +0000 (0:00:00.229) 0:03:07.629 ******** 2026-08-18 14:45:21.221278 | node1 | ok: [node1] => { 2026-08-18 14:45:21.221385 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:45:21.221514 | node1 | } 2026-08-18 14:45:21.221647 | node1 | ok: [node2] => { 2026-08-18 14:45:21.221784 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:45:21.222680 | node1 | } 2026-08-18 14:45:21.222734 | node1 | ok: [node3] => { 2026-08-18 14:45:21.222742 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:45:21.222749 | node1 | } 2026-08-18 14:45:21.222755 | node1 | 2026-08-18 14:45:21.222760 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:45:21.222767 | node1 | Tuesday 18 August 2026 14:45:21 +0000 (0:00:00.072) 0:03:07.701 ******** 2026-08-18 14:45:21.532380 | node1 | ok: [node1] 2026-08-18 14:45:21.532431 | node1 | ok: [node3] 2026-08-18 14:45:21.532525 | node1 | ok: [node2] 2026-08-18 14:45:21.532610 | node1 | 2026-08-18 14:45:21.532708 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:45:21.532805 | node1 | Tuesday 18 August 2026 14:45:21 +0000 (0:00:00.311) 0:03:08.012 ******** 2026-08-18 14:45:21.623161 | node1 | skipping: [node1] 2026-08-18 14:45:21.623654 | node1 | skipping: [node2] 2026-08-18 14:45:21.623709 | node1 | skipping: [node3] 2026-08-18 14:45:21.623714 | node1 | 2026-08-18 14:45:21.623719 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:45:21.623724 | node1 | Tuesday 18 August 2026 14:45:21 +0000 (0:00:00.090) 0:03:08.103 ******** 2026-08-18 14:45:21.728631 | node1 | skipping: [node1] 2026-08-18 14:45:21.728799 | node1 | skipping: [node2] 2026-08-18 14:45:21.728999 | node1 | skipping: [node3] 2026-08-18 14:45:21.729689 | node1 | 2026-08-18 14:45:21.729762 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:45:21.729773 | node1 | Tuesday 18 August 2026 14:45:21 +0000 (0:00:00.104) 0:03:08.207 ******** 2026-08-18 14:45:21.949276 | node1 | ok: [node1] 2026-08-18 14:45:21.949325 | node1 | ok: [node2] 2026-08-18 14:45:21.949329 | node1 | ok: [node3] 2026-08-18 14:45:21.949334 | node1 | 2026-08-18 14:45:21.949338 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:45:21.949343 | node1 | Tuesday 18 August 2026 14:45:21 +0000 (0:00:00.221) 0:03:08.429 ******** 2026-08-18 14:45:23.149789 | node1 | ok: [node3] 2026-08-18 14:45:23.149839 | node1 | ok: [node2] 2026-08-18 14:45:23.149844 | node1 | ok: [node1] 2026-08-18 14:45:23.149848 | node1 | 2026-08-18 14:45:23.149853 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:45:23.149857 | node1 | Tuesday 18 August 2026 14:45:23 +0000 (0:00:01.200) 0:03:09.629 ******** 2026-08-18 14:45:23.235675 | node1 | ok: [node1] => { 2026-08-18 14:45:23.235724 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:45:23.235729 | node1 | } 2026-08-18 14:45:23.235733 | node1 | ok: [node2] => { 2026-08-18 14:45:23.235737 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:45:23.235741 | node1 | } 2026-08-18 14:45:23.235745 | node1 | ok: [node3] => { 2026-08-18 14:45:23.235755 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:45:23.235759 | node1 | } 2026-08-18 14:45:23.235764 | node1 | 2026-08-18 14:45:23.235768 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:45:23.235781 | node1 | Tuesday 18 August 2026 14:45:23 +0000 (0:00:00.085) 0:03:09.714 ******** 2026-08-18 14:45:23.597422 | node1 | ok: [node2] 2026-08-18 14:45:23.597508 | node1 | ok: [node3] 2026-08-18 14:45:23.597519 | node1 | ok: [node1] 2026-08-18 14:45:23.597562 | node1 | 2026-08-18 14:45:23.597667 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:45:23.597764 | node1 | Tuesday 18 August 2026 14:45:23 +0000 (0:00:00.362) 0:03:10.077 ******** 2026-08-18 14:45:25.545422 | node1 | ok: [node1] 2026-08-18 14:45:25.545502 | node1 | ok: [node3] 2026-08-18 14:45:25.545622 | node1 | ok: [node2] 2026-08-18 14:45:25.545736 | node1 | 2026-08-18 14:45:25.545867 | node1 | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-08-18 14:45:25.545999 | node1 | Tuesday 18 August 2026 14:45:25 +0000 (0:00:01.948) 0:03:12.025 ******** 2026-08-18 14:45:25.592526 | node1 | skipping: [node1] 2026-08-18 14:45:25.592732 | node1 | skipping: [node2] 2026-08-18 14:45:25.592926 | node1 | skipping: [node3] 2026-08-18 14:45:25.593128 | node1 | 2026-08-18 14:45:25.593425 | node1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-08-18 14:45:25.593629 | node1 | Tuesday 18 August 2026 14:45:25 +0000 (0:00:00.046) 0:03:12.072 ******** 2026-08-18 14:45:25.621957 | node1 | skipping: [node1] 2026-08-18 14:45:25.653903 | node1 | skipping: [node2] 2026-08-18 14:45:25.654090 | node1 | skipping: [node3] 2026-08-18 14:45:25.654273 | node1 | 2026-08-18 14:45:25.654480 | node1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-08-18 14:45:25.654695 | node1 | Tuesday 18 August 2026 14:45:25 +0000 (0:00:00.061) 0:03:12.133 ******** 2026-08-18 14:45:25.717495 | node1 | skipping: [node1] 2026-08-18 14:45:25.717601 | node1 | skipping: [node2] 2026-08-18 14:45:25.717728 | node1 | skipping: [node3] 2026-08-18 14:45:25.717856 | node1 | 2026-08-18 14:45:25.717979 | node1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-08-18 14:45:25.718101 | node1 | Tuesday 18 August 2026 14:45:25 +0000 (0:00:00.063) 0:03:12.197 ******** 2026-08-18 14:45:26.983426 | node1 | ok: [node2] 2026-08-18 14:45:26.983499 | node1 | ok: [node3] 2026-08-18 14:45:26.983510 | node1 | ok: [node1] 2026-08-18 14:45:26.983519 | node1 | 2026-08-18 14:45:26.983529 | node1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-08-18 14:45:26.983538 | node1 | Tuesday 18 August 2026 14:45:26 +0000 (0:00:01.264) 0:03:13.462 ******** 2026-08-18 14:45:27.454537 | node1 | ok: [node2] 2026-08-18 14:45:27.454624 | node1 | ok: [node1] 2026-08-18 14:45:27.454936 | node1 | ok: [node3] 2026-08-18 14:45:27.454954 | node1 | 2026-08-18 14:45:27.455152 | node1 | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-08-18 14:45:27.455246 | node1 | Tuesday 18 August 2026 14:45:27 +0000 (0:00:00.471) 0:03:13.934 ******** 2026-08-18 14:45:28.390669 | node1 | ok: [node1] => (item={'path': '/etc/containerd'}) 2026-08-18 14:45:28.390763 | node1 | ok: [node3] => (item={'path': '/etc/containerd'}) 2026-08-18 14:45:28.391706 | node1 | ok: [node2] => (item={'path': '/etc/containerd'}) 2026-08-18 14:45:28.391733 | node1 | ok: [node3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:45:28.391744 | node1 | ok: [node2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:45:28.393273 | node1 | ok: [node1] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:45:28.395850 | node1 | ok: [node2] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:45:28.396105 | node1 | ok: [node3] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:45:28.396124 | node1 | ok: [node1] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:45:28.396133 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:45:28.396142 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:45:28.396151 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:45:28.396160 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:45:28.396191 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:45:28.396200 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:45:28.396209 | node1 | 2026-08-18 14:45:28.396219 | node1 | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-08-18 14:45:28.396232 | node1 | Tuesday 18 August 2026 14:45:28 +0000 (0:00:00.936) 0:03:14.870 ******** 2026-08-18 14:45:28.955317 | node1 | ok: [node2] 2026-08-18 14:45:28.955382 | node1 | ok: [node1] 2026-08-18 14:45:28.955387 | node1 | ok: [node3] 2026-08-18 14:45:28.955392 | node1 | 2026-08-18 14:45:28.955404 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:45:28.955568 | node1 | Tuesday 18 August 2026 14:45:28 +0000 (0:00:00.546) 0:03:15.416 ******** 2026-08-18 14:45:28.955794 | node1 | 2026-08-18 14:45:28.956128 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:45:28.956307 | node1 | Tuesday 18 August 2026 14:45:28 +0000 (0:00:00.006) 0:03:15.423 ******** 2026-08-18 14:45:28.956453 | node1 | 2026-08-18 14:45:28.956609 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:45:28.956763 | node1 | Tuesday 18 August 2026 14:45:28 +0000 (0:00:00.005) 0:03:15.429 ******** 2026-08-18 14:45:28.956909 | node1 | 2026-08-18 14:45:28.957063 | node1 | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-08-18 14:45:28.958024 | node1 | Tuesday 18 August 2026 14:45:28 +0000 (0:00:00.005) 0:03:15.434 ******** 2026-08-18 14:45:29.337352 | node1 | ok: [node3] 2026-08-18 14:45:29.337532 | node1 | ok: [node2] 2026-08-18 14:45:29.337727 | node1 | ok: [node1] 2026-08-18 14:45:29.337908 | node1 | 2026-08-18 14:45:29.338139 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:45:29.338342 | node1 | Tuesday 18 August 2026 14:45:29 +0000 (0:00:00.382) 0:03:15.816 ******** 2026-08-18 14:45:29.577633 | node1 | ok: [node1] 2026-08-18 14:45:29.577727 | node1 | ok: [node3] 2026-08-18 14:45:29.577935 | node1 | ok: [node2] 2026-08-18 14:45:29.578137 | node1 | 2026-08-18 14:45:29.578357 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:45:29.578593 | node1 | Tuesday 18 August 2026 14:45:29 +0000 (0:00:00.240) 0:03:16.057 ******** 2026-08-18 14:45:29.656046 | node1 | ok: [node1] => { 2026-08-18 14:45:29.656185 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:45:29.656314 | node1 | } 2026-08-18 14:45:29.656441 | node1 | ok: [node2] => { 2026-08-18 14:45:29.656577 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:45:29.656696 | node1 | } 2026-08-18 14:45:29.656893 | node1 | ok: [node3] => { 2026-08-18 14:45:29.657046 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:45:29.657206 | node1 | } 2026-08-18 14:45:29.657391 | node1 | 2026-08-18 14:45:29.657540 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:45:29.657674 | node1 | Tuesday 18 August 2026 14:45:29 +0000 (0:00:00.078) 0:03:16.135 ******** 2026-08-18 14:45:30.025680 | node1 | ok: [node2] 2026-08-18 14:45:30.025849 | node1 | ok: [node1] 2026-08-18 14:45:30.026045 | node1 | ok: [node3] 2026-08-18 14:45:30.026238 | node1 | 2026-08-18 14:45:30.026446 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:45:30.026653 | node1 | Tuesday 18 August 2026 14:45:30 +0000 (0:00:00.369) 0:03:16.505 ******** 2026-08-18 14:45:34.099142 | node1 | ok: [node1] 2026-08-18 14:45:34.099237 | node1 | ok: [node2] 2026-08-18 14:45:34.099415 | node1 | ok: [node3] 2026-08-18 14:45:34.099652 | node1 | 2026-08-18 14:45:34.099923 | node1 | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-08-18 14:45:34.100168 | node1 | Tuesday 18 August 2026 14:45:34 +0000 (0:00:04.073) 0:03:20.579 ******** 2026-08-18 14:45:35.405187 | node1 | ok: [node2] 2026-08-18 14:45:35.406285 | node1 | ok: [node3] 2026-08-18 14:45:35.406378 | node1 | ok: [node1] 2026-08-18 14:45:35.406391 | node1 | 2026-08-18 14:45:35.406402 | node1 | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-08-18 14:45:35.406412 | node1 | Tuesday 18 August 2026 14:45:35 +0000 (0:00:01.305) 0:03:21.884 ******** 2026-08-18 14:45:35.626756 | node1 | ok: [node1] 2026-08-18 14:45:35.626883 | node1 | ok: [node2] 2026-08-18 14:45:35.627471 | node1 | ok: [node3] 2026-08-18 14:45:35.627531 | node1 | 2026-08-18 14:45:35.627538 | node1 | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-08-18 14:45:35.627545 | node1 | Tuesday 18 August 2026 14:45:35 +0000 (0:00:00.221) 0:03:22.106 ******** 2026-08-18 14:45:36.065539 | node1 | ok: [node1] 2026-08-18 14:45:36.065609 | node1 | ok: [node2] 2026-08-18 14:45:36.065937 | node1 | ok: [node3] 2026-08-18 14:45:36.065953 | node1 | 2026-08-18 14:45:36.065958 | node1 | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-08-18 14:45:36.065963 | node1 | Tuesday 18 August 2026 14:45:36 +0000 (0:00:00.438) 0:03:22.545 ******** 2026-08-18 14:45:36.667713 | node1 | ok: [node1] => (item={'path': '/etc/docker'}) 2026-08-18 14:45:36.667898 | node1 | ok: [node2] => (item={'path': '/etc/docker'}) 2026-08-18 14:45:36.667940 | node1 | ok: [node3] => (item={'path': '/etc/docker'}) 2026-08-18 14:45:36.668335 | node1 | ok: [node1] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:45:36.668404 | node1 | ok: [node2] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:45:36.668423 | node1 | ok: [node3] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:45:36.669190 | node1 | ok: [node1] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:45:36.669251 | node1 | ok: [node2] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:45:36.669261 | node1 | ok: [node3] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:45:36.669271 | node1 | 2026-08-18 14:45:36.669280 | node1 | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-08-18 14:45:36.669300 | node1 | Tuesday 18 August 2026 14:45:36 +0000 (0:00:00.602) 0:03:23.147 ******** 2026-08-18 14:45:37.098665 | node1 | ok: [node3] 2026-08-18 14:45:37.098735 | node1 | ok: [node2] 2026-08-18 14:45:37.099301 | node1 | ok: [node1] 2026-08-18 14:45:37.099351 | node1 | 2026-08-18 14:45:37.099359 | node1 | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-08-18 14:45:37.099366 | node1 | Tuesday 18 August 2026 14:45:37 +0000 (0:00:00.431) 0:03:23.578 ******** 2026-08-18 14:45:37.602363 | node1 | ok: [node1] 2026-08-18 14:45:37.602443 | node1 | ok: [node2] 2026-08-18 14:45:37.602503 | node1 | ok: [node3] 2026-08-18 14:45:37.603573 | node1 | 2026-08-18 14:45:37.603625 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:45:37.603630 | node1 | Tuesday 18 August 2026 14:45:37 +0000 (0:00:00.486) 0:03:24.065 ******** 2026-08-18 14:45:37.603635 | node1 | 2026-08-18 14:45:37.603639 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:45:37.603643 | node1 | Tuesday 18 August 2026 14:45:37 +0000 (0:00:00.006) 0:03:24.072 ******** 2026-08-18 14:45:37.603647 | node1 | 2026-08-18 14:45:37.603651 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:45:37.603655 | node1 | Tuesday 18 August 2026 14:45:37 +0000 (0:00:00.005) 0:03:24.077 ******** 2026-08-18 14:45:37.603659 | node1 | 2026-08-18 14:45:37.603662 | node1 | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-08-18 14:45:37.603666 | node1 | Tuesday 18 August 2026 14:45:37 +0000 (0:00:00.005) 0:03:24.082 ******** 2026-08-18 14:45:37.980852 | node1 | ok: [node1] 2026-08-18 14:45:37.980975 | node1 | ok: [node3] 2026-08-18 14:45:37.981009 | node1 | ok: [node2] 2026-08-18 14:45:37.981162 | node1 | 2026-08-18 14:45:37.981377 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:45:37.981430 | node1 | Tuesday 18 August 2026 14:45:37 +0000 (0:00:00.378) 0:03:24.460 ******** 2026-08-18 14:45:38.041422 | node1 | ok: [node1] => { 2026-08-18 14:45:38.042444 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:45:38.042511 | node1 | } 2026-08-18 14:45:38.042516 | node1 | ok: [node2] => { 2026-08-18 14:45:38.042520 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:45:38.042524 | node1 | } 2026-08-18 14:45:38.042528 | node1 | ok: [node3] => { 2026-08-18 14:45:38.042532 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:45:38.042536 | node1 | } 2026-08-18 14:45:38.042540 | node1 | 2026-08-18 14:45:38.042544 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:45:38.042549 | node1 | Tuesday 18 August 2026 14:45:38 +0000 (0:00:00.060) 0:03:24.521 ******** 2026-08-18 14:45:38.350065 | node1 | ok: [node1] 2026-08-18 14:45:38.350124 | node1 | ok: [node2] 2026-08-18 14:45:38.350133 | node1 | ok: [node3] 2026-08-18 14:45:38.350140 | node1 | 2026-08-18 14:45:38.350147 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:45:38.350153 | node1 | Tuesday 18 August 2026 14:45:38 +0000 (0:00:00.307) 0:03:24.828 ******** 2026-08-18 14:45:38.419702 | node1 | skipping: [node1] 2026-08-18 14:45:38.419873 | node1 | skipping: [node2] 2026-08-18 14:45:38.419974 | node1 | skipping: [node3] 2026-08-18 14:45:38.420054 | node1 | 2026-08-18 14:45:38.420151 | node1 | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-08-18 14:45:38.420243 | node1 | Tuesday 18 August 2026 14:45:38 +0000 (0:00:00.070) 0:03:24.899 ******** 2026-08-18 14:45:38.453039 | node1 | ok: [node1] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/ubuntu-22.04.yml) 2026-08-18 14:45:38.487810 | node1 | ok: [node2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/ubuntu-22.04.yml) 2026-08-18 14:45:38.487985 | node1 | ok: [node3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/ubuntu-22.04.yml) 2026-08-18 14:45:38.488167 | node1 | 2026-08-18 14:45:38.488371 | node1 | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-08-18 14:45:38.488571 | node1 | Tuesday 18 August 2026 14:45:38 +0000 (0:00:00.067) 0:03:24.967 ******** 2026-08-18 14:45:39.761461 | node1 | ok: [node2] 2026-08-18 14:45:39.761581 | node1 | ok: [node3] 2026-08-18 14:45:39.761704 | node1 | ok: [node1] 2026-08-18 14:45:39.761827 | node1 | 2026-08-18 14:45:39.761965 | node1 | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-08-18 14:45:39.762102 | node1 | Tuesday 18 August 2026 14:45:39 +0000 (0:00:01.273) 0:03:26.241 ******** 2026-08-18 14:45:40.439286 | node1 | ok: [node1] => (item=chronyd) 2026-08-18 14:45:40.439404 | node1 | ok: [node3] => (item=chronyd) 2026-08-18 14:45:40.439545 | node1 | ok: [node2] => (item=chronyd) 2026-08-18 14:45:40.439692 | node1 | ok: [node1] => (item=sshd) 2026-08-18 14:45:40.439840 | node1 | ok: [node3] => (item=sshd) 2026-08-18 14:45:40.439980 | node1 | ok: [node2] => (item=sshd) 2026-08-18 14:45:40.440112 | node1 | 2026-08-18 14:45:40.440261 | node1 | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-08-18 14:45:40.440407 | node1 | Tuesday 18 August 2026 14:45:40 +0000 (0:00:00.677) 0:03:26.918 ******** 2026-08-18 14:45:40.660478 | node1 | ok: [node1] 2026-08-18 14:45:40.660552 | node1 | ok: [node2] 2026-08-18 14:45:40.660677 | node1 | ok: [node3] 2026-08-18 14:45:40.660794 | node1 | 2026-08-18 14:45:40.660927 | node1 | TASK [vexxhost.ceph.cephadm : Ensure the cephadm SSH user exists] ************** 2026-08-18 14:45:40.661060 | node1 | Tuesday 18 August 2026 14:45:40 +0000 (0:00:00.221) 0:03:27.140 ******** 2026-08-18 14:45:40.919434 | node1 | ok: [node1] 2026-08-18 14:45:40.919496 | node1 | ok: [node2] 2026-08-18 14:45:40.919507 | node1 | ok: [node3] 2026-08-18 14:45:40.919517 | node1 | 2026-08-18 14:45:40.919527 | node1 | TASK [vexxhost.ceph.cephadm : Grant passwordless sudo to the cephadm SSH user] *** 2026-08-18 14:45:40.919536 | node1 | Tuesday 18 August 2026 14:45:40 +0000 (0:00:00.257) 0:03:27.398 ******** 2026-08-18 14:45:41.347111 | node1 | ok: [node1] 2026-08-18 14:45:41.347188 | node1 | ok: [node2] 2026-08-18 14:45:41.347202 | node1 | ok: [node3] 2026-08-18 14:45:41.347535 | node1 | 2026-08-18 14:45:41.347590 | node1 | TASK [vexxhost.ceph.cephadm : Log in to the container registry] **************** 2026-08-18 14:45:41.347595 | node1 | Tuesday 18 August 2026 14:45:41 +0000 (0:00:00.428) 0:03:27.827 ******** 2026-08-18 14:45:41.377245 | node1 | skipping: [node1] 2026-08-18 14:45:41.408222 | node1 | skipping: [node2] 2026-08-18 14:45:41.408244 | node1 | skipping: [node3] 2026-08-18 14:45:41.408250 | node1 | 2026-08-18 14:45:41.408257 | node1 | TASK [vexxhost.ceph.cephadm : Pre-pull the Ceph image] ************************* 2026-08-18 14:45:41.408263 | node1 | Tuesday 18 August 2026 14:45:41 +0000 (0:00:00.061) 0:03:27.888 ******** 2026-08-18 14:45:41.774653 | node1 | ok: [node1] 2026-08-18 14:45:41.774701 | node1 | ok: [node2] 2026-08-18 14:45:41.774706 | node1 | ok: [node3] 2026-08-18 14:45:41.774710 | node1 | 2026-08-18 14:45:41.774714 | node1 | TASK [vexxhost.ceph.cephadm : Pin CEPHADM_IMAGE for interactive login shells] *** 2026-08-18 14:45:41.774719 | node1 | Tuesday 18 August 2026 14:45:41 +0000 (0:00:00.366) 0:03:28.254 ******** 2026-08-18 14:45:42.215846 | node1 | ok: [node1] 2026-08-18 14:45:42.215917 | node1 | ok: [node3] 2026-08-18 14:45:42.215931 | node1 | ok: [node2] 2026-08-18 14:45:42.216270 | node1 | 2026-08-18 14:45:42.216329 | node1 | TASK [vexxhost.ceph.osd : Locate the cluster admin host] *********************** 2026-08-18 14:45:42.216334 | node1 | Tuesday 18 August 2026 14:45:42 +0000 (0:00:00.441) 0:03:28.695 ******** 2026-08-18 14:45:42.781040 | node1 | ok: [node1] => (item=node1) 2026-08-18 14:45:42.781131 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 14:45:42.781524 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 14:45:42.781580 | node1 | 2026-08-18 14:45:42.781585 | node1 | TASK [vexxhost.ceph.osd : Select the admin host and its mon config path] ******* 2026-08-18 14:45:42.781590 | node1 | Tuesday 18 August 2026 14:45:42 +0000 (0:00:00.565) 0:03:29.261 ******** 2026-08-18 14:45:42.850133 | node1 | ok: [node1] 2026-08-18 14:45:42.850227 | node1 | ok: [node2] 2026-08-18 14:45:42.850714 | node1 | ok: [node3] 2026-08-18 14:45:42.850774 | node1 | 2026-08-18 14:45:42.850781 | node1 | TASK [vexxhost.ceph.osd : Set the admin mon config path] *********************** 2026-08-18 14:45:42.850788 | node1 | Tuesday 18 August 2026 14:45:42 +0000 (0:00:00.068) 0:03:29.330 ******** 2026-08-18 14:45:42.909320 | node1 | ok: [node1] 2026-08-18 14:45:42.909763 | node1 | ok: [node2] 2026-08-18 14:45:42.909863 | node1 | ok: [node3] 2026-08-18 14:45:42.909874 | node1 | 2026-08-18 14:45:42.909884 | node1 | TASK [Register this OSD host] ************************************************** 2026-08-18 14:45:42.909904 | node1 | Tuesday 18 August 2026 14:45:42 +0000 (0:00:00.059) 0:03:29.389 ******** 2026-08-18 14:45:43.003853 | node1 | included: vexxhost.ceph.cephadm_host for node1, node2, node3 2026-08-18 14:45:43.003892 | node1 | 2026-08-18 14:45:43.004149 | node1 | TASK [vexxhost.ceph.cephadm_host : Get the cephadm public key] ***************** 2026-08-18 14:45:43.004202 | node1 | Tuesday 18 August 2026 14:45:43 +0000 (0:00:00.094) 0:03:29.484 ******** 2026-08-18 14:45:44.411893 | node1 | ok: [node1] 2026-08-18 14:45:44.411982 | node1 | 2026-08-18 14:45:44.412007 | node1 | TASK [vexxhost.ceph.cephadm_host : Resolve the cluster SSH public key to authorise] *** 2026-08-18 14:45:44.412150 | node1 | Tuesday 18 August 2026 14:45:44 +0000 (0:00:01.407) 0:03:30.891 ******** 2026-08-18 14:45:44.477156 | node1 | ok: [node1] 2026-08-18 14:45:44.477211 | node1 | ok: [node2] 2026-08-18 14:45:44.477585 | node1 | ok: [node3] 2026-08-18 14:45:44.477639 | node1 | 2026-08-18 14:45:44.477645 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert a cluster SSH key was resolved] ****** 2026-08-18 14:45:44.477651 | node1 | Tuesday 18 August 2026 14:45:44 +0000 (0:00:00.065) 0:03:30.957 ******** 2026-08-18 14:45:44.535180 | node1 | ok: [node1] => { 2026-08-18 14:45:44.536949 | node1 | "changed": false, 2026-08-18 14:45:44.537014 | node1 | "msg": "All assertions passed" 2026-08-18 14:45:44.537025 | node1 | } 2026-08-18 14:45:44.537033 | node1 | ok: [node2] => { 2026-08-18 14:45:44.537040 | node1 | "changed": false, 2026-08-18 14:45:44.537047 | node1 | "msg": "All assertions passed" 2026-08-18 14:45:44.537054 | node1 | } 2026-08-18 14:45:44.537060 | node1 | ok: [node3] => { 2026-08-18 14:45:44.537065 | node1 | "changed": false, 2026-08-18 14:45:44.537077 | node1 | "msg": "All assertions passed" 2026-08-18 14:45:44.537084 | node1 | } 2026-08-18 14:45:44.537143 | node1 | 2026-08-18 14:45:44.537151 | node1 | TASK [vexxhost.ceph.cephadm_host : Authorise the cluster SSH key for the cephadm user] *** 2026-08-18 14:45:44.537174 | node1 | Tuesday 18 August 2026 14:45:44 +0000 (0:00:00.057) 0:03:31.015 ******** 2026-08-18 14:45:44.825942 | node1 | ok: [node1] 2026-08-18 14:45:44.826017 | node1 | ok: [node3] 2026-08-18 14:45:44.826407 | node1 | ok: [node2] 2026-08-18 14:45:44.826458 | node1 | 2026-08-18 14:45:44.826464 | node1 | TASK [vexxhost.ceph.cephadm_host : Derive this host's cephadm labels from inventory membership] *** 2026-08-18 14:45:44.826469 | node1 | Tuesday 18 August 2026 14:45:44 +0000 (0:00:00.291) 0:03:31.306 ******** 2026-08-18 14:45:44.892997 | node1 | ok: [node1] 2026-08-18 14:45:44.893173 | node1 | ok: [node2] 2026-08-18 14:45:44.893670 | node1 | ok: [node3] 2026-08-18 14:45:44.893694 | node1 | 2026-08-18 14:45:44.893701 | node1 | TASK [vexxhost.ceph.cephadm_host : Find this host's addresses on the public network] *** 2026-08-18 14:45:44.893708 | node1 | Tuesday 18 August 2026 14:45:44 +0000 (0:00:00.066) 0:03:31.372 ******** 2026-08-18 14:45:44.991563 | node1 | ok: [node1] 2026-08-18 14:45:44.991666 | node1 | ok: [node2] 2026-08-18 14:45:44.991680 | node1 | ok: [node3] 2026-08-18 14:45:44.992055 | node1 | 2026-08-18 14:45:44.992102 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert exactly one address matches the public network] *** 2026-08-18 14:45:44.992109 | node1 | Tuesday 18 August 2026 14:45:44 +0000 (0:00:00.098) 0:03:31.471 ******** 2026-08-18 14:45:45.062555 | node1 | ok: [node1] => { 2026-08-18 14:45:45.064448 | node1 | "changed": false, 2026-08-18 14:45:45.064502 | node1 | "msg": "All assertions passed" 2026-08-18 14:45:45.064510 | node1 | } 2026-08-18 14:45:45.064516 | node1 | ok: [node2] => { 2026-08-18 14:45:45.064522 | node1 | "changed": false, 2026-08-18 14:45:45.064528 | node1 | "msg": "All assertions passed" 2026-08-18 14:45:45.064534 | node1 | } 2026-08-18 14:45:45.064539 | node1 | ok: [node3] => { 2026-08-18 14:45:45.064545 | node1 | "changed": false, 2026-08-18 14:45:45.064551 | node1 | "msg": "All assertions passed" 2026-08-18 14:45:45.064557 | node1 | } 2026-08-18 14:45:45.064563 | node1 | 2026-08-18 14:45:45.064570 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host (labels)] ***************** 2026-08-18 14:45:45.064578 | node1 | Tuesday 18 August 2026 14:45:45 +0000 (0:00:00.070) 0:03:31.542 ******** 2026-08-18 14:45:46.580237 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:45:46.580308 | node1 | ok: [node1] 2026-08-18 14:45:46.580704 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:45:46.580755 | node1 | 2026-08-18 14:45:46.580760 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host with a CRUSH location] **** 2026-08-18 14:45:46.580765 | node1 | Tuesday 18 August 2026 14:45:46 +0000 (0:00:01.517) 0:03:33.060 ******** 2026-08-18 14:45:46.660754 | node1 | skipping: [node1] 2026-08-18 14:45:46.660772 | node1 | skipping: [node2] 2026-08-18 14:45:46.660777 | node1 | skipping: [node3] 2026-08-18 14:45:46.660781 | node1 | 2026-08-18 14:45:46.660785 | node1 | TASK [vexxhost.ceph.osd : Apply OSD service specs (drive groups)] ************** 2026-08-18 14:45:46.660789 | node1 | Tuesday 18 August 2026 14:45:46 +0000 (0:00:00.080) 0:03:33.140 ******** 2026-08-18 14:45:46.682883 | node1 | skipping: [node1] 2026-08-18 14:45:46.682900 | node1 | 2026-08-18 14:45:46.682905 | node1 | TASK [vexxhost.ceph.osd : List existing ceph-volume OSDs on this host] ********* 2026-08-18 14:45:46.682909 | node1 | Tuesday 18 August 2026 14:45:46 +0000 (0:00:00.022) 0:03:33.163 ******** 2026-08-18 14:45:47.917288 | node1 | ok: [node3] 2026-08-18 14:45:47.917379 | node1 | ok: [node1] 2026-08-18 14:45:47.917823 | node1 | ok: [node2] 2026-08-18 14:45:47.917881 | node1 | 2026-08-18 14:45:47.917888 | node1 | TASK [vexxhost.ceph.osd : Deploy OSDs on the explicitly listed devices] ******** 2026-08-18 14:45:47.917894 | node1 | Tuesday 18 August 2026 14:45:47 +0000 (0:00:01.233) 0:03:34.397 ******** 2026-08-18 14:49:30.085508 | node1 | changed: [node1] => (item=/dev/ceph-node1-osd0/data) 2026-08-18 14:49:30.085642 | node1 | changed: [node2 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node2-osd0/data) 2026-08-18 14:49:30.085783 | node1 | changed: [node3 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node3-osd0/data) 2026-08-18 14:49:30.085915 | node1 | changed: [node1] => (item=/dev/ceph-node1-osd1/data) 2026-08-18 14:49:30.086052 | node1 | changed: [node2 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node2-osd1/data) 2026-08-18 14:49:30.086205 | node1 | changed: [node3 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node3-osd1/data) 2026-08-18 14:49:30.086354 | node1 | changed: [node1] => (item=/dev/ceph-node1-osd2/data) 2026-08-18 14:49:30.086500 | node1 | changed: [node2 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node2-osd2/data) 2026-08-18 14:49:30.086636 | node1 | changed: [node3 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node3-osd2/data) 2026-08-18 14:49:30.086725 | node1 | 2026-08-18 14:49:30.086823 | node1 | TASK [vexxhost.ceph.osd : Wait for explicitly requested OSD daemons to be running on this host] *** 2026-08-18 14:49:30.086916 | node1 | Tuesday 18 August 2026 14:49:30 +0000 (0:03:42.157) 0:07:16.555 ******** 2026-08-18 14:49:44.591395 | node1 | ok: [node1] 2026-08-18 14:49:44.591522 | node1 | FAILED - RETRYING: [node3 -> node1]: Wait for explicitly requested OSD daemons to be running on this host (120 retries left). 2026-08-18 14:49:44.591659 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:49:44.591802 | node1 | FAILED - RETRYING: [node3 -> node1]: Wait for explicitly requested OSD daemons to be running on this host (119 retries left). 2026-08-18 14:49:44.591934 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:49:44.592053 | node1 | 2026-08-18 14:49:44.592191 | node1 | TASK [vexxhost.ceph.osd : Wait for OSD drive-group services to deploy at least one daemon] *** 2026-08-18 14:49:44.592326 | node1 | Tuesday 18 August 2026 14:49:44 +0000 (0:00:14.516) 0:07:31.071 ******** 2026-08-18 14:49:44.670751 | node1 | skipping: [node1] 2026-08-18 14:49:44.671063 | node1 | 2026-08-18 14:49:44.671275 | node1 | PLAY [Apply Ceph configuration] ************************************************ 2026-08-18 14:49:44.671457 | node1 | 2026-08-18 14:49:44.671658 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:49:44.671862 | node1 | Tuesday 18 August 2026 14:49:44 +0000 (0:00:00.079) 0:07:31.151 ******** 2026-08-18 14:49:45.748394 | node1 | ok: [node1] 2026-08-18 14:49:45.748597 | node1 | ok: [node3] 2026-08-18 14:49:45.748816 | node1 | ok: [node2] 2026-08-18 14:49:45.749051 | node1 | 2026-08-18 14:49:45.749296 | node1 | TASK [vexxhost.ceph.configure : Locate the cluster admin host] ***************** 2026-08-18 14:49:45.749516 | node1 | Tuesday 18 August 2026 14:49:45 +0000 (0:00:01.077) 0:07:32.228 ******** 2026-08-18 14:49:46.306140 | node1 | ok: [node1] => (item=node1) 2026-08-18 14:49:46.306215 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 14:49:46.306278 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 14:49:46.306338 | node1 | 2026-08-18 14:49:46.306435 | node1 | TASK [vexxhost.ceph.configure : Select the admin host] ************************* 2026-08-18 14:49:46.306529 | node1 | Tuesday 18 August 2026 14:49:46 +0000 (0:00:00.557) 0:07:32.785 ******** 2026-08-18 14:49:46.365297 | node1 | ok: [node1] 2026-08-18 14:49:46.365504 | node1 | ok: [node2] 2026-08-18 14:49:46.365745 | node1 | ok: [node3] 2026-08-18 14:49:46.365933 | node1 | 2026-08-18 14:49:46.366233 | node1 | TASK [vexxhost.ceph.configure : Set the admin mon config path] ***************** 2026-08-18 14:49:46.366512 | node1 | Tuesday 18 August 2026 14:49:46 +0000 (0:00:00.058) 0:07:32.844 ******** 2026-08-18 14:49:46.419129 | node1 | ok: [node1] 2026-08-18 14:49:46.419329 | node1 | ok: [node2] 2026-08-18 14:49:46.419629 | node1 | ok: [node3] 2026-08-18 14:49:46.419841 | node1 | 2026-08-18 14:49:46.420074 | node1 | TASK [vexxhost.ceph.configure : Apply central Ceph configuration] ************** 2026-08-18 14:49:46.420303 | node1 | Tuesday 18 August 2026 14:49:46 +0000 (0:00:00.054) 0:07:32.899 ******** 2026-08-18 14:49:46.486977 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/config.yml for node1, node2, node3 2026-08-18 14:49:46.487145 | node1 | 2026-08-18 14:49:46.487389 | node1 | TASK [vexxhost.ceph.configure : Apply central Ceph configuration] ************** 2026-08-18 14:49:46.487609 | node1 | Tuesday 18 August 2026 14:49:46 +0000 (0:00:00.067) 0:07:32.967 ******** 2026-08-18 14:50:03.462977 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/standby_behaviour) 2026-08-18 14:50:03.463155 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/RGW_API_SSL_VERIFY) 2026-08-18 14:50:03.463376 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/FEATURE_TOGGLE_CEPHFS) 2026-08-18 14:50:03.464277 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/FEATURE_TOGGLE_ISCSI) 2026-08-18 14:50:03.464289 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/FEATURE_TOGGLE_MIRRORING) 2026-08-18 14:50:03.464295 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/FEATURE_TOGGLE_NFS) 2026-08-18 14:50:03.464301 | node1 | 2026-08-18 14:50:03.464341 | node1 | TASK [vexxhost.ceph.configure : Configure erasure-code profiles] *************** 2026-08-18 14:50:03.464546 | node1 | Tuesday 18 August 2026 14:50:03 +0000 (0:00:16.975) 0:07:49.942 ******** 2026-08-18 14:50:03.527849 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/ec_profiles.yml for node1, node2, node3 2026-08-18 14:50:03.528020 | node1 | 2026-08-18 14:50:03.528341 | node1 | TASK [vexxhost.ceph.configure : Configure erasure-code profiles] *************** 2026-08-18 14:50:03.528672 | node1 | Tuesday 18 August 2026 14:50:03 +0000 (0:00:00.064) 0:07:50.007 ******** 2026-08-18 14:50:07.477357 | node1 | changed: [node1] => (item=ec_hdd_s3) 2026-08-18 14:50:07.477443 | node1 | 2026-08-18 14:50:07.477569 | node1 | TASK [vexxhost.ceph.configure : Configure CRUSH rules] ************************* 2026-08-18 14:50:07.477718 | node1 | Tuesday 18 August 2026 14:50:07 +0000 (0:00:03.949) 0:07:53.957 ******** 2026-08-18 14:50:07.535757 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/crush_rules.yml for node1, node2, node3 2026-08-18 14:50:07.535830 | node1 | 2026-08-18 14:50:07.535889 | node1 | TASK [vexxhost.ceph.configure : Configure CRUSH rules] ************************* 2026-08-18 14:50:07.536065 | node1 | Tuesday 18 August 2026 14:50:07 +0000 (0:00:00.058) 0:07:54.015 ******** 2026-08-18 14:50:16.686240 | node1 | changed: [node1] => (item=rep_hdd) 2026-08-18 14:50:16.686341 | node1 | changed: [node1] => (item=rep_lab) 2026-08-18 14:50:16.686454 | node1 | changed: [node1] => (item=ec_hdd_s3) 2026-08-18 14:50:16.686572 | node1 | 2026-08-18 14:50:16.686711 | node1 | TASK [vexxhost.ceph.configure : Configure pools] ******************************* 2026-08-18 14:50:16.686841 | node1 | Tuesday 18 August 2026 14:50:16 +0000 (0:00:09.150) 0:08:03.166 ******** 2026-08-18 14:50:16.742141 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/pools.yml for node1, node2, node3 2026-08-18 14:50:16.742357 | node1 | 2026-08-18 14:50:16.742578 | node1 | TASK [vexxhost.ceph.configure : Configure pools] ******************************* 2026-08-18 14:50:16.742788 | node1 | Tuesday 18 August 2026 14:50:16 +0000 (0:00:00.056) 0:08:03.222 ******** 2026-08-18 14:50:47.507501 | node1 | changed: [node1] => (item=molecule-1.rgw.buckets.data) 2026-08-18 14:50:47.507558 | node1 | changed: [node1] => (item=molecule-1.rgw.buckets.index) 2026-08-18 14:50:47.507569 | node1 | changed: [node1] => (item=molecule-1.rgw.buckets.non-ec) 2026-08-18 14:50:47.507579 | node1 | changed: [node1] => (item=molecule-1.rgw.control) 2026-08-18 14:50:47.507589 | node1 | changed: [node1] => (item=molecule-1.rgw.log) 2026-08-18 14:50:47.507599 | node1 | changed: [node1] => (item=molecule-1.rgw.meta) 2026-08-18 14:50:47.507609 | node1 | 2026-08-18 14:50:47.507619 | node1 | TASK [vexxhost.ceph.configure : Configure CephX keys] ************************** 2026-08-18 14:50:47.507629 | node1 | Tuesday 18 August 2026 14:50:47 +0000 (0:00:30.762) 0:08:33.985 ******** 2026-08-18 14:50:47.557279 | node1 | skipping: [node1] 2026-08-18 14:50:47.557353 | node1 | skipping: [node2] 2026-08-18 14:50:47.557884 | node1 | skipping: [node3] 2026-08-18 14:50:47.557905 | node1 | 2026-08-18 14:50:47.557912 | node1 | TASK [vexxhost.ceph.configure : Enroll the dashboard TLS certificate] ********** 2026-08-18 14:50:47.557919 | node1 | Tuesday 18 August 2026 14:50:47 +0000 (0:00:00.051) 0:08:34.036 ******** 2026-08-18 14:50:47.628121 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/dashboard.yml for node1, node2, node3 2026-08-18 14:50:47.628161 | node1 | 2026-08-18 14:50:47.628389 | node1 | TASK [vexxhost.ceph.configure : Check whether the dashboard mgr module is enabled] *** 2026-08-18 14:50:47.628432 | node1 | Tuesday 18 August 2026 14:50:47 +0000 (0:00:00.071) 0:08:34.108 ******** 2026-08-18 14:50:49.178744 | node1 | ok: [node1] 2026-08-18 14:50:49.178827 | node1 | 2026-08-18 14:50:49.179233 | node1 | TASK [vexxhost.ceph.configure : Get the current dashboard certificate] ********* 2026-08-18 14:50:49.179261 | node1 | Tuesday 18 August 2026 14:50:49 +0000 (0:00:01.550) 0:08:35.658 ******** 2026-08-18 14:50:50.739116 | node1 | ok: [node1] 2026-08-18 14:50:50.739229 | node1 | 2026-08-18 14:50:50.739297 | node1 | TASK [vexxhost.ceph.configure : Set the dashboard certificate] ***************** 2026-08-18 14:50:50.739482 | node1 | Tuesday 18 August 2026 14:50:50 +0000 (0:00:01.560) 0:08:37.219 ******** 2026-08-18 14:50:50.777321 | node1 | skipping: [node1] 2026-08-18 14:50:50.777397 | node1 | 2026-08-18 14:50:50.777410 | node1 | TASK [vexxhost.ceph.configure : Set the dashboard certificate key] ************* 2026-08-18 14:50:50.777512 | node1 | Tuesday 18 August 2026 14:50:50 +0000 (0:00:00.038) 0:08:37.257 ******** 2026-08-18 14:50:50.801058 | node1 | skipping: [node1] 2026-08-18 14:50:50.801166 | node1 | 2026-08-18 14:50:50.801387 | node1 | TASK [vexxhost.ceph.configure : Flag the dashboard for reload] ***************** 2026-08-18 14:50:50.801434 | node1 | Tuesday 18 August 2026 14:50:50 +0000 (0:00:00.023) 0:08:37.281 ******** 2026-08-18 14:50:50.838601 | node1 | skipping: [node1] 2026-08-18 14:50:50.838715 | node1 | 2026-08-18 14:50:50.839029 | node1 | TASK [vexxhost.ceph.configure : Create a self-signed certificate] ************** 2026-08-18 14:50:50.839079 | node1 | Tuesday 18 August 2026 14:50:50 +0000 (0:00:00.037) 0:08:37.318 ******** 2026-08-18 14:50:52.529833 | node1 | changed: [node1] 2026-08-18 14:50:52.529920 | node1 | 2026-08-18 14:50:52.530175 | node1 | TASK [vexxhost.ceph.configure : Flag the dashboard for reload] ***************** 2026-08-18 14:50:52.530401 | node1 | Tuesday 18 August 2026 14:50:52 +0000 (0:00:01.691) 0:08:39.009 ******** 2026-08-18 14:50:52.580096 | node1 | ok: [node1] 2026-08-18 14:50:52.580153 | node1 | 2026-08-18 14:50:52.580364 | node1 | TASK [vexxhost.ceph.configure : Reload the dashboard so it picks up the certificate] *** 2026-08-18 14:50:52.580382 | node1 | Tuesday 18 August 2026 14:50:52 +0000 (0:00:00.050) 0:08:39.060 ******** 2026-08-18 14:50:59.278477 | node1 | changed: [node1] => (item=ceph mgr module disable dashboard) 2026-08-18 14:50:59.278530 | node1 | changed: [node1] => (item=ceph mgr module enable dashboard) 2026-08-18 14:50:59.278537 | node1 | changed: [node1] => (item=ceph orch reconfig mgr) 2026-08-18 14:50:59.278544 | node1 | 2026-08-18 14:50:59.278550 | node1 | TASK [vexxhost.ceph.configure : Configure dashboard users] ********************* 2026-08-18 14:50:59.278556 | node1 | Tuesday 18 August 2026 14:50:59 +0000 (0:00:06.697) 0:08:45.757 ******** 2026-08-18 14:50:59.354020 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/dashboard_users.yml for node1, node2, node3 2026-08-18 14:50:59.354067 | node1 | 2026-08-18 14:50:59.354072 | node1 | TASK [vexxhost.ceph.configure : Configure dashboard users] ********************* 2026-08-18 14:50:59.354077 | node1 | Tuesday 18 August 2026 14:50:59 +0000 (0:00:00.076) 0:08:45.833 ******** 2026-08-18 14:51:08.000604 | node1 | changed: [node1] => (item=None) 2026-08-18 14:51:08.000683 | node1 | changed: [node1] => (item=None) 2026-08-18 14:51:08.000786 | node1 | changed: [node1 -> {{ ceph_configure_admin_host }}] 2026-08-18 14:51:08.000876 | node1 | 2026-08-18 14:51:08.000975 | node1 | PLAY [Deploy Ceph RADOS Gateways] ********************************************** 2026-08-18 14:51:08.001177 | node1 | 2026-08-18 14:51:08.001237 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:51:08.001282 | node1 | Tuesday 18 August 2026 14:51:07 +0000 (0:00:08.647) 0:08:54.481 ******** 2026-08-18 14:51:09.181721 | node1 | ok: [node1] 2026-08-18 14:51:09.181850 | node1 | ok: [node2] 2026-08-18 14:51:09.182549 | node1 | ok: [node3] 2026-08-18 14:51:09.182646 | node1 | 2026-08-18 14:51:09.182908 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:51:09.183010 | node1 | Tuesday 18 August 2026 14:51:09 +0000 (0:00:01.180) 0:08:55.662 ******** 2026-08-18 14:51:09.440883 | node1 | ok: [node1] 2026-08-18 14:51:09.441076 | node1 | ok: [node2] 2026-08-18 14:51:09.441310 | node1 | ok: [node3] 2026-08-18 14:51:09.441522 | node1 | 2026-08-18 14:51:09.441733 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:51:09.441937 | node1 | Tuesday 18 August 2026 14:51:09 +0000 (0:00:00.258) 0:08:55.920 ******** 2026-08-18 14:51:09.511164 | node1 | skipping: [node1] 2026-08-18 14:51:09.511350 | node1 | skipping: [node2] 2026-08-18 14:51:09.511544 | node1 | skipping: [node3] 2026-08-18 14:51:09.511724 | node1 | 2026-08-18 14:51:09.511963 | node1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-08-18 14:51:09.512167 | node1 | Tuesday 18 August 2026 14:51:09 +0000 (0:00:00.070) 0:08:55.991 ******** 2026-08-18 14:51:09.752670 | node1 | ok: [node1] 2026-08-18 14:51:09.752860 | node1 | ok: [node3] 2026-08-18 14:51:09.753058 | node1 | ok: [node2] 2026-08-18 14:51:09.753312 | node1 | 2026-08-18 14:51:09.753533 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:51:09.753737 | node1 | Tuesday 18 August 2026 14:51:09 +0000 (0:00:00.241) 0:08:56.232 ******** 2026-08-18 14:51:09.832154 | node1 | ok: [node1] => { 2026-08-18 14:51:09.832404 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:51:09.833168 | node1 | } 2026-08-18 14:51:09.833211 | node1 | ok: [node2] => { 2026-08-18 14:51:09.833223 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:51:09.833330 | node1 | } 2026-08-18 14:51:09.834596 | node1 | ok: [node3] => { 2026-08-18 14:51:09.834937 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:51:09.835146 | node1 | } 2026-08-18 14:51:09.835331 | node1 | 2026-08-18 14:51:09.835554 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:51:09.835746 | node1 | Tuesday 18 August 2026 14:51:09 +0000 (0:00:00.079) 0:08:56.312 ******** 2026-08-18 14:51:10.170684 | node1 | ok: [node1] 2026-08-18 14:51:10.170891 | node1 | ok: [node2] 2026-08-18 14:51:10.171112 | node1 | ok: [node3] 2026-08-18 14:51:10.171318 | node1 | 2026-08-18 14:51:10.171553 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:51:10.171771 | node1 | Tuesday 18 August 2026 14:51:10 +0000 (0:00:00.338) 0:08:56.650 ******** 2026-08-18 14:51:10.254474 | node1 | skipping: [node1] 2026-08-18 14:51:10.254677 | node1 | skipping: [node2] 2026-08-18 14:51:10.254873 | node1 | skipping: [node3] 2026-08-18 14:51:10.255057 | node1 | 2026-08-18 14:51:10.255268 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:51:10.255470 | node1 | Tuesday 18 August 2026 14:51:10 +0000 (0:00:00.083) 0:08:56.734 ******** 2026-08-18 14:51:10.338116 | node1 | skipping: [node1] 2026-08-18 14:51:10.338329 | node1 | skipping: [node2] 2026-08-18 14:51:10.338535 | node1 | skipping: [node3] 2026-08-18 14:51:10.338723 | node1 | 2026-08-18 14:51:10.338931 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:51:10.339133 | node1 | Tuesday 18 August 2026 14:51:10 +0000 (0:00:00.083) 0:08:56.818 ******** 2026-08-18 14:51:10.593755 | node1 | ok: [node1] 2026-08-18 14:51:10.593923 | node1 | ok: [node2] 2026-08-18 14:51:10.594121 | node1 | ok: [node3] 2026-08-18 14:51:10.594306 | node1 | 2026-08-18 14:51:10.594516 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:51:10.594717 | node1 | Tuesday 18 August 2026 14:51:10 +0000 (0:00:00.255) 0:08:57.073 ******** 2026-08-18 14:51:11.844071 | node1 | ok: [node2] 2026-08-18 14:51:11.844261 | node1 | ok: [node1] 2026-08-18 14:51:11.844501 | node1 | ok: [node3] 2026-08-18 14:51:11.844703 | node1 | 2026-08-18 14:51:11.844937 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:51:11.845205 | node1 | Tuesday 18 August 2026 14:51:11 +0000 (0:00:01.250) 0:08:58.324 ******** 2026-08-18 14:51:11.920242 | node1 | ok: [node1] => { 2026-08-18 14:51:11.920498 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:51:11.920713 | node1 | } 2026-08-18 14:51:11.920930 | node1 | ok: [node2] => { 2026-08-18 14:51:11.921232 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:51:11.921488 | node1 | } 2026-08-18 14:51:11.921718 | node1 | ok: [node3] => { 2026-08-18 14:51:11.921956 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:51:11.922170 | node1 | } 2026-08-18 14:51:11.922374 | node1 | 2026-08-18 14:51:11.922619 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:51:11.922846 | node1 | Tuesday 18 August 2026 14:51:11 +0000 (0:00:00.076) 0:08:58.400 ******** 2026-08-18 14:51:12.290664 | node1 | ok: [node1] 2026-08-18 14:51:12.290852 | node1 | ok: [node2] 2026-08-18 14:51:12.291063 | node1 | ok: [node3] 2026-08-18 14:51:12.291268 | node1 | 2026-08-18 14:51:12.291524 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:51:12.291744 | node1 | Tuesday 18 August 2026 14:51:12 +0000 (0:00:00.369) 0:08:58.770 ******** 2026-08-18 14:51:14.315757 | node1 | ok: [node1] 2026-08-18 14:51:14.315813 | node1 | ok: [node2] 2026-08-18 14:51:14.315825 | node1 | ok: [node3] 2026-08-18 14:51:14.315835 | node1 | 2026-08-18 14:51:14.315846 | node1 | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-08-18 14:51:14.315861 | node1 | Tuesday 18 August 2026 14:51:14 +0000 (0:00:02.024) 0:09:00.794 ******** 2026-08-18 14:51:14.366631 | node1 | skipping: [node1] 2026-08-18 14:51:14.366755 | node1 | skipping: [node2] 2026-08-18 14:51:14.366880 | node1 | skipping: [node3] 2026-08-18 14:51:14.367041 | node1 | 2026-08-18 14:51:14.367211 | node1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-08-18 14:51:14.367355 | node1 | Tuesday 18 August 2026 14:51:14 +0000 (0:00:00.051) 0:09:00.846 ******** 2026-08-18 14:51:14.399919 | node1 | skipping: [node1] 2026-08-18 14:51:14.433360 | node1 | skipping: [node2] 2026-08-18 14:51:14.433427 | node1 | skipping: [node3] 2026-08-18 14:51:14.433476 | node1 | 2026-08-18 14:51:14.433675 | node1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-08-18 14:51:14.433694 | node1 | Tuesday 18 August 2026 14:51:14 +0000 (0:00:00.067) 0:09:00.913 ******** 2026-08-18 14:51:14.459546 | node1 | skipping: [node1] 2026-08-18 14:51:14.487042 | node1 | skipping: [node2] 2026-08-18 14:51:14.487127 | node1 | skipping: [node3] 2026-08-18 14:51:14.487236 | node1 | 2026-08-18 14:51:14.487367 | node1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-08-18 14:51:14.487493 | node1 | Tuesday 18 August 2026 14:51:14 +0000 (0:00:00.053) 0:09:00.967 ******** 2026-08-18 14:51:15.822468 | node1 | ok: [node2] 2026-08-18 14:51:15.822551 | node1 | ok: [node1] 2026-08-18 14:51:15.822694 | node1 | ok: [node3] 2026-08-18 14:51:15.822818 | node1 | 2026-08-18 14:51:15.822962 | node1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-08-18 14:51:15.823097 | node1 | Tuesday 18 August 2026 14:51:15 +0000 (0:00:01.335) 0:09:02.302 ******** 2026-08-18 14:51:16.283450 | node1 | ok: [node1] 2026-08-18 14:51:16.283522 | node1 | ok: [node2] 2026-08-18 14:51:16.283913 | node1 | ok: [node3] 2026-08-18 14:51:16.283958 | node1 | 2026-08-18 14:51:16.283963 | node1 | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-08-18 14:51:16.283968 | node1 | Tuesday 18 August 2026 14:51:16 +0000 (0:00:00.461) 0:09:02.763 ******** 2026-08-18 14:51:17.204447 | node1 | ok: [node1] => (item={'path': '/etc/containerd'}) 2026-08-18 14:51:17.204496 | node1 | ok: [node2] => (item={'path': '/etc/containerd'}) 2026-08-18 14:51:17.204501 | node1 | ok: [node3] => (item={'path': '/etc/containerd'}) 2026-08-18 14:51:17.204506 | node1 | ok: [node1] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:51:17.204511 | node1 | ok: [node2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:51:17.204515 | node1 | ok: [node3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:51:17.204519 | node1 | ok: [node2] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:51:17.204523 | node1 | ok: [node3] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:51:17.204537 | node1 | ok: [node1] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:51:17.204541 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:51:17.204545 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:51:17.204549 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:51:17.204553 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:51:17.204557 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:51:17.204561 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:51:17.204565 | node1 | 2026-08-18 14:51:17.204570 | node1 | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-08-18 14:51:17.204573 | node1 | Tuesday 18 August 2026 14:51:17 +0000 (0:00:00.919) 0:09:03.683 ******** 2026-08-18 14:51:17.765170 | node1 | ok: [node1] 2026-08-18 14:51:17.765216 | node1 | ok: [node2] 2026-08-18 14:51:17.765223 | node1 | ok: [node3] 2026-08-18 14:51:17.765229 | node1 | 2026-08-18 14:51:17.765240 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:51:17.765591 | node1 | Tuesday 18 August 2026 14:51:17 +0000 (0:00:00.526) 0:09:04.209 ******** 2026-08-18 14:51:17.765984 | node1 | 2026-08-18 14:51:17.766006 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:51:17.766015 | node1 | Tuesday 18 August 2026 14:51:17 +0000 (0:00:00.013) 0:09:04.223 ******** 2026-08-18 14:51:17.766024 | node1 | 2026-08-18 14:51:17.766033 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:51:17.766044 | node1 | Tuesday 18 August 2026 14:51:17 +0000 (0:00:00.011) 0:09:04.234 ******** 2026-08-18 14:51:17.766455 | node1 | 2026-08-18 14:51:17.766515 | node1 | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-08-18 14:51:17.766522 | node1 | Tuesday 18 August 2026 14:51:17 +0000 (0:00:00.010) 0:09:04.245 ******** 2026-08-18 14:51:18.160198 | node1 | ok: [node2] 2026-08-18 14:51:18.160265 | node1 | ok: [node1] 2026-08-18 14:51:18.160402 | node1 | ok: [node3] 2026-08-18 14:51:18.160657 | node1 | 2026-08-18 14:51:18.160675 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:51:18.160680 | node1 | Tuesday 18 August 2026 14:51:18 +0000 (0:00:00.395) 0:09:04.640 ******** 2026-08-18 14:51:18.396970 | node1 | ok: [node2] 2026-08-18 14:51:18.397034 | node1 | ok: [node1] 2026-08-18 14:51:18.397256 | node1 | ok: [node3] 2026-08-18 14:51:18.397372 | node1 | 2026-08-18 14:51:18.397386 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:51:18.397403 | node1 | Tuesday 18 August 2026 14:51:18 +0000 (0:00:00.236) 0:09:04.877 ******** 2026-08-18 14:51:18.476632 | node1 | ok: [node1] => { 2026-08-18 14:51:18.476733 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:51:18.476809 | node1 | } 2026-08-18 14:51:18.477235 | node1 | ok: [node2] => { 2026-08-18 14:51:18.477314 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:51:18.477327 | node1 | } 2026-08-18 14:51:18.477344 | node1 | ok: [node3] => { 2026-08-18 14:51:18.477939 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:51:18.477990 | node1 | } 2026-08-18 14:51:18.477997 | node1 | 2026-08-18 14:51:18.478004 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:51:18.478011 | node1 | Tuesday 18 August 2026 14:51:18 +0000 (0:00:00.079) 0:09:04.956 ******** 2026-08-18 14:51:18.852237 | node1 | ok: [node2] 2026-08-18 14:51:18.852310 | node1 | ok: [node1] 2026-08-18 14:51:18.852662 | node1 | ok: [node3] 2026-08-18 14:51:18.852710 | node1 | 2026-08-18 14:51:18.852722 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:51:18.852850 | node1 | Tuesday 18 August 2026 14:51:18 +0000 (0:00:00.375) 0:09:05.332 ******** 2026-08-18 14:51:21.990564 | node1 | ok: [node1] 2026-08-18 14:51:21.990633 | node1 | ok: [node3] 2026-08-18 14:51:21.990713 | node1 | ok: [node2] 2026-08-18 14:51:21.990987 | node1 | 2026-08-18 14:51:21.991002 | node1 | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-08-18 14:51:21.991008 | node1 | Tuesday 18 August 2026 14:51:21 +0000 (0:00:03.138) 0:09:08.470 ******** 2026-08-18 14:51:23.081562 | node1 | ok: [node1] 2026-08-18 14:51:23.081766 | node1 | ok: [node2] 2026-08-18 14:51:23.082149 | node1 | ok: [node3] 2026-08-18 14:51:23.082195 | node1 | 2026-08-18 14:51:23.082200 | node1 | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-08-18 14:51:23.082205 | node1 | Tuesday 18 August 2026 14:51:23 +0000 (0:00:01.091) 0:09:09.561 ******** 2026-08-18 14:51:23.304887 | node1 | ok: [node1] 2026-08-18 14:51:23.304943 | node1 | ok: [node2] 2026-08-18 14:51:23.305211 | node1 | ok: [node3] 2026-08-18 14:51:23.305260 | node1 | 2026-08-18 14:51:23.305272 | node1 | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-08-18 14:51:23.305379 | node1 | Tuesday 18 August 2026 14:51:23 +0000 (0:00:00.223) 0:09:09.785 ******** 2026-08-18 14:51:23.746689 | node1 | ok: [node1] 2026-08-18 14:51:23.746772 | node1 | ok: [node2] 2026-08-18 14:51:23.747360 | node1 | ok: [node3] 2026-08-18 14:51:23.747418 | node1 | 2026-08-18 14:51:23.747426 | node1 | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-08-18 14:51:23.747434 | node1 | Tuesday 18 August 2026 14:51:23 +0000 (0:00:00.441) 0:09:10.226 ******** 2026-08-18 14:51:24.322590 | node1 | ok: [node1] => (item={'path': '/etc/docker'}) 2026-08-18 14:51:24.322631 | node1 | ok: [node2] => (item={'path': '/etc/docker'}) 2026-08-18 14:51:24.322636 | node1 | ok: [node3] => (item={'path': '/etc/docker'}) 2026-08-18 14:51:24.322640 | node1 | ok: [node1] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:51:24.322646 | node1 | ok: [node2] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:51:24.322650 | node1 | ok: [node3] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:51:24.322654 | node1 | ok: [node1] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:51:24.322658 | node1 | ok: [node2] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:51:24.322662 | node1 | ok: [node3] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:51:24.322666 | node1 | 2026-08-18 14:51:24.322670 | node1 | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-08-18 14:51:24.322674 | node1 | Tuesday 18 August 2026 14:51:24 +0000 (0:00:00.575) 0:09:10.801 ******** 2026-08-18 14:51:24.762237 | node1 | ok: [node1] 2026-08-18 14:51:24.762923 | node1 | ok: [node2] 2026-08-18 14:51:24.762970 | node1 | ok: [node3] 2026-08-18 14:51:24.762978 | node1 | 2026-08-18 14:51:24.762985 | node1 | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-08-18 14:51:24.762992 | node1 | Tuesday 18 August 2026 14:51:24 +0000 (0:00:00.440) 0:09:11.242 ******** 2026-08-18 14:51:25.256239 | node1 | ok: [node1] 2026-08-18 14:51:25.256307 | node1 | ok: [node2] 2026-08-18 14:51:25.256320 | node1 | ok: [node3] 2026-08-18 14:51:25.257186 | node1 | 2026-08-18 14:51:25.257253 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:51:25.257264 | node1 | Tuesday 18 August 2026 14:51:25 +0000 (0:00:00.451) 0:09:11.693 ******** 2026-08-18 14:51:25.257274 | node1 | 2026-08-18 14:51:25.257282 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:51:25.257291 | node1 | Tuesday 18 August 2026 14:51:25 +0000 (0:00:00.019) 0:09:11.713 ******** 2026-08-18 14:51:25.257299 | node1 | 2026-08-18 14:51:25.257308 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:51:25.257316 | node1 | Tuesday 18 August 2026 14:51:25 +0000 (0:00:00.011) 0:09:11.725 ******** 2026-08-18 14:51:25.257332 | node1 | 2026-08-18 14:51:25.257341 | node1 | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-08-18 14:51:25.257427 | node1 | Tuesday 18 August 2026 14:51:25 +0000 (0:00:00.011) 0:09:11.736 ******** 2026-08-18 14:51:25.659473 | node1 | ok: [node1] 2026-08-18 14:51:25.659543 | node1 | ok: [node3] 2026-08-18 14:51:25.659896 | node1 | ok: [node2] 2026-08-18 14:51:25.659938 | node1 | 2026-08-18 14:51:25.659944 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:51:25.659949 | node1 | Tuesday 18 August 2026 14:51:25 +0000 (0:00:00.403) 0:09:12.139 ******** 2026-08-18 14:51:25.722652 | node1 | ok: [node1] => { 2026-08-18 14:51:25.724035 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:51:25.724083 | node1 | } 2026-08-18 14:51:25.724090 | node1 | ok: [node2] => { 2026-08-18 14:51:25.724096 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:51:25.724102 | node1 | } 2026-08-18 14:51:25.724107 | node1 | ok: [node3] => { 2026-08-18 14:51:25.724112 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:51:25.724117 | node1 | } 2026-08-18 14:51:25.724123 | node1 | 2026-08-18 14:51:25.724129 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:51:25.724140 | node1 | Tuesday 18 August 2026 14:51:25 +0000 (0:00:00.063) 0:09:12.202 ******** 2026-08-18 14:51:26.018041 | node1 | ok: [node1] 2026-08-18 14:51:26.018104 | node1 | ok: [node2] 2026-08-18 14:51:26.018191 | node1 | ok: [node3] 2026-08-18 14:51:26.018466 | node1 | 2026-08-18 14:51:26.018482 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:51:26.018487 | node1 | Tuesday 18 August 2026 14:51:26 +0000 (0:00:00.295) 0:09:12.498 ******** 2026-08-18 14:51:26.084937 | node1 | skipping: [node1] 2026-08-18 14:51:26.084999 | node1 | skipping: [node2] 2026-08-18 14:51:26.085082 | node1 | skipping: [node3] 2026-08-18 14:51:26.085381 | node1 | 2026-08-18 14:51:26.085397 | node1 | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-08-18 14:51:26.085402 | node1 | Tuesday 18 August 2026 14:51:26 +0000 (0:00:00.067) 0:09:12.565 ******** 2026-08-18 14:51:26.116020 | node1 | ok: [node1] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/ubuntu-22.04.yml) 2026-08-18 14:51:26.159575 | node1 | ok: [node2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/ubuntu-22.04.yml) 2026-08-18 14:51:26.159672 | node1 | ok: [node3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/ubuntu-22.04.yml) 2026-08-18 14:51:26.159786 | node1 | 2026-08-18 14:51:26.160048 | node1 | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-08-18 14:51:26.160068 | node1 | Tuesday 18 August 2026 14:51:26 +0000 (0:00:00.074) 0:09:12.639 ******** 2026-08-18 14:51:27.382409 | node1 | ok: [node1] 2026-08-18 14:51:27.382524 | node1 | ok: [node3] 2026-08-18 14:51:27.383034 | node1 | ok: [node2] 2026-08-18 14:51:27.383080 | node1 | 2026-08-18 14:51:27.383085 | node1 | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-08-18 14:51:27.383090 | node1 | Tuesday 18 August 2026 14:51:27 +0000 (0:00:01.222) 0:09:13.862 ******** 2026-08-18 14:51:28.096894 | node1 | ok: [node1] => (item=chronyd) 2026-08-18 14:51:28.097013 | node1 | ok: [node2] => (item=chronyd) 2026-08-18 14:51:28.097941 | node1 | ok: [node3] => (item=chronyd) 2026-08-18 14:51:28.098025 | node1 | ok: [node1] => (item=sshd) 2026-08-18 14:51:28.098032 | node1 | ok: [node2] => (item=sshd) 2026-08-18 14:51:28.098038 | node1 | ok: [node3] => (item=sshd) 2026-08-18 14:51:28.098044 | node1 | 2026-08-18 14:51:28.098050 | node1 | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-08-18 14:51:28.098056 | node1 | Tuesday 18 August 2026 14:51:28 +0000 (0:00:00.714) 0:09:14.576 ******** 2026-08-18 14:51:28.317582 | node1 | ok: [node1] 2026-08-18 14:51:28.317656 | node1 | ok: [node2] 2026-08-18 14:51:28.318377 | node1 | ok: [node3] 2026-08-18 14:51:28.318427 | node1 | 2026-08-18 14:51:28.318434 | node1 | TASK [vexxhost.ceph.cephadm : Ensure the cephadm SSH user exists] ************** 2026-08-18 14:51:28.318441 | node1 | Tuesday 18 August 2026 14:51:28 +0000 (0:00:00.220) 0:09:14.797 ******** 2026-08-18 14:51:28.583708 | node1 | ok: [node1] 2026-08-18 14:51:28.583787 | node1 | ok: [node2] 2026-08-18 14:51:28.584284 | node1 | ok: [node3] 2026-08-18 14:51:28.584332 | node1 | 2026-08-18 14:51:28.584339 | node1 | TASK [vexxhost.ceph.cephadm : Grant passwordless sudo to the cephadm SSH user] *** 2026-08-18 14:51:28.584346 | node1 | Tuesday 18 August 2026 14:51:28 +0000 (0:00:00.266) 0:09:15.063 ******** 2026-08-18 14:51:29.026522 | node1 | ok: [node1] 2026-08-18 14:51:29.026601 | node1 | ok: [node3] 2026-08-18 14:51:29.026606 | node1 | ok: [node2] 2026-08-18 14:51:29.026611 | node1 | 2026-08-18 14:51:29.026617 | node1 | TASK [vexxhost.ceph.cephadm : Log in to the container registry] **************** 2026-08-18 14:51:29.026622 | node1 | Tuesday 18 August 2026 14:51:29 +0000 (0:00:00.442) 0:09:15.505 ******** 2026-08-18 14:51:29.091068 | node1 | skipping: [node1] 2026-08-18 14:51:29.091127 | node1 | skipping: [node2] 2026-08-18 14:51:29.091133 | node1 | skipping: [node3] 2026-08-18 14:51:29.091137 | node1 | 2026-08-18 14:51:29.091142 | node1 | TASK [vexxhost.ceph.cephadm : Pre-pull the Ceph image] ************************* 2026-08-18 14:51:29.091147 | node1 | Tuesday 18 August 2026 14:51:29 +0000 (0:00:00.064) 0:09:15.570 ******** 2026-08-18 14:51:29.461820 | node1 | ok: [node1] 2026-08-18 14:51:29.461936 | node1 | ok: [node2] 2026-08-18 14:51:29.461950 | node1 | ok: [node3] 2026-08-18 14:51:29.462395 | node1 | 2026-08-18 14:51:29.462464 | node1 | TASK [vexxhost.ceph.cephadm : Pin CEPHADM_IMAGE for interactive login shells] *** 2026-08-18 14:51:29.462472 | node1 | Tuesday 18 August 2026 14:51:29 +0000 (0:00:00.371) 0:09:15.941 ******** 2026-08-18 14:51:29.941761 | node1 | ok: [node1] 2026-08-18 14:51:29.941839 | node1 | ok: [node2] 2026-08-18 14:51:29.941854 | node1 | ok: [node3] 2026-08-18 14:51:29.942197 | node1 | 2026-08-18 14:51:29.942274 | node1 | TASK [vexxhost.ceph.rgw : Locate the cluster admin host] *********************** 2026-08-18 14:51:29.942281 | node1 | Tuesday 18 August 2026 14:51:29 +0000 (0:00:00.479) 0:09:16.421 ******** 2026-08-18 14:51:30.548107 | node1 | ok: [node1] => (item=node1) 2026-08-18 14:51:30.548154 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 14:51:30.548160 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 14:51:30.548165 | node1 | 2026-08-18 14:51:30.548169 | node1 | TASK [vexxhost.ceph.rgw : Select the admin host] ******************************* 2026-08-18 14:51:30.548173 | node1 | Tuesday 18 August 2026 14:51:30 +0000 (0:00:00.606) 0:09:17.027 ******** 2026-08-18 14:51:30.615382 | node1 | ok: [node1] 2026-08-18 14:51:30.615431 | node1 | ok: [node2] 2026-08-18 14:51:30.615437 | node1 | ok: [node3] 2026-08-18 14:51:30.615444 | node1 | 2026-08-18 14:51:30.615449 | node1 | TASK [vexxhost.ceph.rgw : Set the admin mon config path] *********************** 2026-08-18 14:51:30.615455 | node1 | Tuesday 18 August 2026 14:51:30 +0000 (0:00:00.066) 0:09:17.094 ******** 2026-08-18 14:51:30.678637 | node1 | ok: [node1] 2026-08-18 14:51:30.678686 | node1 | ok: [node2] 2026-08-18 14:51:30.678692 | node1 | ok: [node3] 2026-08-18 14:51:30.678698 | node1 | 2026-08-18 14:51:30.678704 | node1 | TASK [Register this RGW host] ************************************************** 2026-08-18 14:51:30.678710 | node1 | Tuesday 18 August 2026 14:51:30 +0000 (0:00:00.063) 0:09:17.158 ******** 2026-08-18 14:51:30.777701 | node1 | included: vexxhost.ceph.cephadm_host for node1, node2, node3 2026-08-18 14:51:30.777744 | node1 | 2026-08-18 14:51:30.777749 | node1 | TASK [vexxhost.ceph.cephadm_host : Get the cephadm public key] ***************** 2026-08-18 14:51:30.777754 | node1 | Tuesday 18 August 2026 14:51:30 +0000 (0:00:00.099) 0:09:17.257 ******** 2026-08-18 14:51:32.171119 | node1 | ok: [node1] 2026-08-18 14:51:32.171173 | node1 | 2026-08-18 14:51:32.171181 | node1 | TASK [vexxhost.ceph.cephadm_host : Resolve the cluster SSH public key to authorise] *** 2026-08-18 14:51:32.171188 | node1 | Tuesday 18 August 2026 14:51:32 +0000 (0:00:01.392) 0:09:18.650 ******** 2026-08-18 14:51:32.228803 | node1 | ok: [node1] 2026-08-18 14:51:32.228875 | node1 | ok: [node2] 2026-08-18 14:51:32.228945 | node1 | ok: [node3] 2026-08-18 14:51:32.229247 | node1 | 2026-08-18 14:51:32.229309 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert a cluster SSH key was resolved] ****** 2026-08-18 14:51:32.229327 | node1 | Tuesday 18 August 2026 14:51:32 +0000 (0:00:00.058) 0:09:18.709 ******** 2026-08-18 14:51:32.294228 | node1 | ok: [node1] => { 2026-08-18 14:51:32.294334 | node1 | "changed": false, 2026-08-18 14:51:32.295926 | node1 | "msg": "All assertions passed" 2026-08-18 14:51:32.295972 | node1 | } 2026-08-18 14:51:32.295979 | node1 | ok: [node2] => { 2026-08-18 14:51:32.295985 | node1 | "changed": false, 2026-08-18 14:51:32.295991 | node1 | "msg": "All assertions passed" 2026-08-18 14:51:32.295997 | node1 | } 2026-08-18 14:51:32.296002 | node1 | ok: [node3] => { 2026-08-18 14:51:32.296007 | node1 | "changed": false, 2026-08-18 14:51:32.296013 | node1 | "msg": "All assertions passed" 2026-08-18 14:51:32.296033 | node1 | } 2026-08-18 14:51:32.296039 | node1 | 2026-08-18 14:51:32.296045 | node1 | TASK [vexxhost.ceph.cephadm_host : Authorise the cluster SSH key for the cephadm user] *** 2026-08-18 14:51:32.296051 | node1 | Tuesday 18 August 2026 14:51:32 +0000 (0:00:00.065) 0:09:18.774 ******** 2026-08-18 14:51:32.589938 | node1 | ok: [node1] 2026-08-18 14:51:32.590086 | node1 | ok: [node2] 2026-08-18 14:51:32.590104 | node1 | ok: [node3] 2026-08-18 14:51:32.590526 | node1 | 2026-08-18 14:51:32.590616 | node1 | TASK [vexxhost.ceph.cephadm_host : Derive this host's cephadm labels from inventory membership] *** 2026-08-18 14:51:32.590626 | node1 | Tuesday 18 August 2026 14:51:32 +0000 (0:00:00.295) 0:09:19.069 ******** 2026-08-18 14:51:32.668490 | node1 | ok: [node1] 2026-08-18 14:51:32.668602 | node1 | ok: [node2] 2026-08-18 14:51:32.668612 | node1 | ok: [node3] 2026-08-18 14:51:32.668965 | node1 | 2026-08-18 14:51:32.669038 | node1 | TASK [vexxhost.ceph.cephadm_host : Find this host's addresses on the public network] *** 2026-08-18 14:51:32.669045 | node1 | Tuesday 18 August 2026 14:51:32 +0000 (0:00:00.078) 0:09:19.147 ******** 2026-08-18 14:51:32.771199 | node1 | ok: [node1] 2026-08-18 14:51:32.771344 | node1 | ok: [node2] 2026-08-18 14:51:32.771407 | node1 | ok: [node3] 2026-08-18 14:51:32.771845 | node1 | 2026-08-18 14:51:32.771908 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert exactly one address matches the public network] *** 2026-08-18 14:51:32.771917 | node1 | Tuesday 18 August 2026 14:51:32 +0000 (0:00:00.102) 0:09:19.250 ******** 2026-08-18 14:51:32.846376 | node1 | ok: [node1] => { 2026-08-18 14:51:32.848364 | node1 | "changed": false, 2026-08-18 14:51:32.848433 | node1 | "msg": "All assertions passed" 2026-08-18 14:51:32.848442 | node1 | } 2026-08-18 14:51:32.848451 | node1 | ok: [node2] => { 2026-08-18 14:51:32.848458 | node1 | "changed": false, 2026-08-18 14:51:32.848466 | node1 | "msg": "All assertions passed" 2026-08-18 14:51:32.848472 | node1 | } 2026-08-18 14:51:32.848477 | node1 | ok: [node3] => { 2026-08-18 14:51:32.848482 | node1 | "changed": false, 2026-08-18 14:51:32.848488 | node1 | "msg": "All assertions passed" 2026-08-18 14:51:32.848493 | node1 | } 2026-08-18 14:51:32.848510 | node1 | 2026-08-18 14:51:32.848517 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host (labels)] ***************** 2026-08-18 14:51:32.848524 | node1 | Tuesday 18 August 2026 14:51:32 +0000 (0:00:00.075) 0:09:19.326 ******** 2026-08-18 14:51:34.423269 | node1 | ok: [node1] 2026-08-18 14:51:34.423401 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:51:34.423679 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:51:34.423730 | node1 | 2026-08-18 14:51:34.423743 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host with a CRUSH location] **** 2026-08-18 14:51:34.423924 | node1 | Tuesday 18 August 2026 14:51:34 +0000 (0:00:01.577) 0:09:20.903 ******** 2026-08-18 14:51:34.508529 | node1 | skipping: [node1] 2026-08-18 14:51:34.508614 | node1 | skipping: [node2] 2026-08-18 14:51:34.509013 | node1 | skipping: [node3] 2026-08-18 14:51:34.509062 | node1 | 2026-08-18 14:51:34.509067 | node1 | TASK [vexxhost.ceph.rgw : Configure RGW multisite (realm, zonegroup, zone)] **** 2026-08-18 14:51:34.509072 | node1 | Tuesday 18 August 2026 14:51:34 +0000 (0:00:00.085) 0:09:20.988 ******** 2026-08-18 14:51:34.555753 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/rgw/tasks/multisite.yml for node1 2026-08-18 14:51:34.555845 | node1 | 2026-08-18 14:51:34.555988 | node1 | TASK [vexxhost.ceph.rgw : Configure RGW realms] ******************************** 2026-08-18 14:51:34.556063 | node1 | Tuesday 18 August 2026 14:51:34 +0000 (0:00:00.047) 0:09:21.036 ******** 2026-08-18 14:51:38.512788 | node1 | changed: [node1] => (item=molecule) 2026-08-18 14:51:38.513142 | node1 | 2026-08-18 14:51:38.513194 | node1 | TASK [vexxhost.ceph.rgw : Configure RGW zonegroups] **************************** 2026-08-18 14:51:38.513210 | node1 | Tuesday 18 August 2026 14:51:38 +0000 (0:00:03.956) 0:09:24.992 ******** 2026-08-18 14:51:40.450253 | node1 | changed: [node1] => (item=molecule) 2026-08-18 14:51:40.450320 | node1 | 2026-08-18 14:51:40.450412 | node1 | TASK [vexxhost.ceph.rgw : Configure RGW zones] ********************************* 2026-08-18 14:51:40.450518 | node1 | Tuesday 18 August 2026 14:51:40 +0000 (0:00:01.938) 0:09:26.930 ******** 2026-08-18 14:51:42.352660 | node1 | changed: [node1] => (item=None) 2026-08-18 14:51:42.352833 | node1 | changed: [node1 -> {{ ceph_rgw_admin_host }}] 2026-08-18 14:51:42.352960 | node1 | 2026-08-18 14:51:42.353190 | node1 | TASK [vexxhost.ceph.rgw : Commit the RGW period] ******************************* 2026-08-18 14:51:42.353327 | node1 | Tuesday 18 August 2026 14:51:42 +0000 (0:00:01.902) 0:09:28.832 ******** 2026-08-18 14:51:43.719955 | node1 | changed: [node1] 2026-08-18 14:51:43.720135 | node1 | 2026-08-18 14:51:43.720346 | node1 | TASK [vexxhost.ceph.rgw : Apply the RGW service specs] ************************* 2026-08-18 14:51:43.720548 | node1 | Tuesday 18 August 2026 14:51:43 +0000 (0:00:01.366) 0:09:30.199 ******** 2026-08-18 14:51:43.767523 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/rgw/tasks/services.yml for node1 2026-08-18 14:51:43.767687 | node1 | 2026-08-18 14:51:43.767905 | node1 | TASK [vexxhost.ceph.rgw : Snapshot current RGW service specs] ****************** 2026-08-18 14:51:43.768099 | node1 | Tuesday 18 August 2026 14:51:43 +0000 (0:00:00.048) 0:09:30.247 ******** 2026-08-18 14:51:45.178479 | node1 | ok: [node1] 2026-08-18 14:51:45.178539 | node1 | 2026-08-18 14:51:45.178550 | node1 | TASK [vexxhost.ceph.rgw : Index current RGW specs by service name] ************* 2026-08-18 14:51:45.178560 | node1 | Tuesday 18 August 2026 14:51:45 +0000 (0:00:01.409) 0:09:31.657 ******** 2026-08-18 14:51:45.215947 | node1 | ok: [node1] 2026-08-18 14:51:45.216038 | node1 | 2026-08-18 14:51:45.216277 | node1 | TASK [vexxhost.ceph.rgw : Apply the RGW service specs] ************************* 2026-08-18 14:51:45.216326 | node1 | Tuesday 18 August 2026 14:51:45 +0000 (0:00:00.038) 0:09:31.696 ******** 2026-08-18 14:51:47.942077 | node1 | changed: [node1] => (item=molecule) 2026-08-18 14:51:47.942175 | node1 | 2026-08-18 14:51:47.942317 | node1 | TASK [vexxhost.ceph.rgw : Redeploy RGW services whose redeploy-triggering keys changed] *** 2026-08-18 14:51:47.942491 | node1 | Tuesday 18 August 2026 14:51:47 +0000 (0:00:02.725) 0:09:34.422 ******** 2026-08-18 14:51:49.427661 | node1 | failed: [node1] (item=rgw.molecule) => {"ansible_loop_var": "item", "changed": true, "cmd": ["cephadm", "shell", "--fsid", "1dff0e0f-3c44-48da-81cd-4f3c6e8722b2", "--config", "/var/lib/ceph/1dff0e0f-3c44-48da-81cd-4f3c6e8722b2/mon.node1/config", "--", "ceph", "orch", "redeploy", "rgw.molecule"], "delta": "0:00:01.272315", "end": "2026-08-18 14:51:49.400344", "item": {"count_per_host": 1, "id": "molecule", "networks": ["0.0.0.0/0"], "spec": {"generate_cert": true, "rgw_frontend_port": 8080, "rgw_realm": "molecule", "rgw_zone": "molecule-1", "rgw_zonegroup": "molecule", "ssl": true}}, "msg": "non-zero return code", "rc": 22, "start": "2026-08-18 14:51:48.128029", "stderr": "Error EINVAL: No daemons exist under service name \"rgw.molecule\". View currently running services using \"ceph orch ls\"", "stderr_lines": ["Error EINVAL: No daemons exist under service name \"rgw.molecule\". View currently running services using \"ceph orch ls\""], "stdout": "", "stdout_lines": []} 2026-08-18 14:51:49.444180 | node1 | 2026-08-18 14:51:49.444289 | node1 | TASK [vexxhost.ceph.rgw : Remove the default zone / zonegroup / pools] ********* 2026-08-18 14:51:49.444389 | node1 | Tuesday 18 August 2026 14:51:49 +0000 (0:00:01.501) 0:09:35.923 ******** 2026-08-18 14:51:49.504659 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/rgw/tasks/cleanup.yml for node2 2026-08-18 14:51:49.504729 | node1 | 2026-08-18 14:51:49.504743 | node1 | TASK [vexxhost.ceph.rgw : List RGW zones] ************************************** 2026-08-18 14:51:49.504856 | node1 | Tuesday 18 August 2026 14:51:49 +0000 (0:00:00.060) 0:09:35.984 ******** 2026-08-18 14:51:50.560331 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:51:50.560412 | node1 | 2026-08-18 14:51:50.560511 | node1 | TASK [vexxhost.ceph.rgw : Remove the default zonegroup] ************************ 2026-08-18 14:51:50.560611 | node1 | Tuesday 18 August 2026 14:51:50 +0000 (0:00:01.056) 0:09:37.040 ******** 2026-08-18 14:51:50.601557 | node1 | skipping: [node2] 2026-08-18 14:51:50.601663 | node1 | 2026-08-18 14:51:50.601789 | node1 | TASK [vexxhost.ceph.rgw : Remove the default zone] ***************************** 2026-08-18 14:51:50.601921 | node1 | Tuesday 18 August 2026 14:51:50 +0000 (0:00:00.041) 0:09:37.081 ******** 2026-08-18 14:51:50.644390 | node1 | skipping: [node2] 2026-08-18 14:51:50.644428 | node1 | 2026-08-18 14:51:50.644544 | node1 | TASK [vexxhost.ceph.rgw : Get the current pool deletion setting] *************** 2026-08-18 14:51:50.644634 | node1 | Tuesday 18 August 2026 14:51:50 +0000 (0:00:00.042) 0:09:37.124 ******** 2026-08-18 14:51:50.687803 | node1 | skipping: [node2] 2026-08-18 14:51:50.687839 | node1 | 2026-08-18 14:51:50.687954 | node1 | TASK [vexxhost.ceph.rgw : Allow pool deletion] ********************************* 2026-08-18 14:51:50.688046 | node1 | Tuesday 18 August 2026 14:51:50 +0000 (0:00:00.043) 0:09:37.168 ******** 2026-08-18 14:51:50.729469 | node1 | skipping: [node2] 2026-08-18 14:51:50.729564 | node1 | 2026-08-18 14:51:50.729696 | node1 | TASK [vexxhost.ceph.rgw : Remove the default RGW pools] ************************ 2026-08-18 14:51:50.729827 | node1 | Tuesday 18 August 2026 14:51:50 +0000 (0:00:00.041) 0:09:37.209 ******** 2026-08-18 14:51:50.793914 | node1 | skipping: [node2] => (item=default.rgw.control) 2026-08-18 14:51:50.794023 | node1 | skipping: [node2] => (item=default.rgw.log) 2026-08-18 14:51:50.794161 | node1 | skipping: [node2] => (item=default.rgw.meta) 2026-08-18 14:51:50.794288 | node1 | skipping: [node2] 2026-08-18 14:51:50.794405 | node1 | 2026-08-18 14:51:50.794537 | node1 | TASK [vexxhost.ceph.rgw : Restore the configured pool deletion setting] ******** 2026-08-18 14:51:50.794659 | node1 | Tuesday 18 August 2026 14:51:50 +0000 (0:00:00.064) 0:09:37.274 ******** 2026-08-18 14:51:50.844083 | node1 | skipping: [node2] 2026-08-18 14:51:50.844143 | node1 | 2026-08-18 14:51:50.844243 | node1 | TASK [vexxhost.ceph.rgw : Remove the temporary pool deletion setting] ********** 2026-08-18 14:51:50.844357 | node1 | Tuesday 18 August 2026 14:51:50 +0000 (0:00:00.050) 0:09:37.324 ******** 2026-08-18 14:51:50.933739 | node1 | skipping: [node2] 2026-08-18 14:51:50.933804 | node1 | 2026-08-18 14:51:50.933907 | node1 | PLAY [Deploy Ceph monitoring stack] ******************************************** 2026-08-18 14:51:50.934220 | node1 | 2026-08-18 14:51:50.934267 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:51:50.934272 | node1 | Tuesday 18 August 2026 14:51:50 +0000 (0:00:00.089) 0:09:37.414 ******** 2026-08-18 14:51:51.859541 | node1 | ok: [node3] 2026-08-18 14:51:51.859607 | node1 | ok: [node2] 2026-08-18 14:51:51.859690 | node1 | 2026-08-18 14:51:51.859922 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:51:51.859967 | node1 | Tuesday 18 August 2026 14:51:51 +0000 (0:00:00.925) 0:09:38.340 ******** 2026-08-18 14:51:52.088236 | node1 | ok: [node2] 2026-08-18 14:51:52.088296 | node1 | ok: [node3] 2026-08-18 14:51:52.088384 | node1 | 2026-08-18 14:51:52.088608 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:51:52.088647 | node1 | Tuesday 18 August 2026 14:51:52 +0000 (0:00:00.228) 0:09:38.568 ******** 2026-08-18 14:51:52.164793 | node1 | skipping: [node2] 2026-08-18 14:51:52.164849 | node1 | skipping: [node3] 2026-08-18 14:51:52.165193 | node1 | 2026-08-18 14:51:52.165233 | node1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-08-18 14:51:52.165239 | node1 | Tuesday 18 August 2026 14:51:52 +0000 (0:00:00.076) 0:09:38.644 ******** 2026-08-18 14:51:52.394848 | node1 | ok: [node2] 2026-08-18 14:51:52.394929 | node1 | ok: [node3] 2026-08-18 14:51:52.394991 | node1 | 2026-08-18 14:51:52.395199 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:51:52.395242 | node1 | Tuesday 18 August 2026 14:51:52 +0000 (0:00:00.230) 0:09:38.874 ******** 2026-08-18 14:51:52.472505 | node1 | ok: [node2] => { 2026-08-18 14:51:52.472569 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:51:52.473295 | node1 | } 2026-08-18 14:51:52.473343 | node1 | ok: [node3] => { 2026-08-18 14:51:52.473348 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:51:52.473353 | node1 | } 2026-08-18 14:51:52.473357 | node1 | 2026-08-18 14:51:52.473362 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:51:52.473366 | node1 | Tuesday 18 August 2026 14:51:52 +0000 (0:00:00.077) 0:09:38.952 ******** 2026-08-18 14:51:52.808732 | node1 | ok: [node2] 2026-08-18 14:51:52.808846 | node1 | ok: [node3] 2026-08-18 14:51:52.809310 | node1 | 2026-08-18 14:51:52.809556 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:51:52.809809 | node1 | Tuesday 18 August 2026 14:51:52 +0000 (0:00:00.335) 0:09:39.288 ******** 2026-08-18 14:51:52.878880 | node1 | skipping: [node2] 2026-08-18 14:51:52.879087 | node1 | skipping: [node3] 2026-08-18 14:51:52.879299 | node1 | 2026-08-18 14:51:52.879530 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:51:52.879766 | node1 | Tuesday 18 August 2026 14:51:52 +0000 (0:00:00.070) 0:09:39.358 ******** 2026-08-18 14:51:52.944346 | node1 | skipping: [node2] 2026-08-18 14:51:52.944462 | node1 | skipping: [node3] 2026-08-18 14:51:52.944596 | node1 | 2026-08-18 14:51:52.944801 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:51:52.944995 | node1 | Tuesday 18 August 2026 14:51:52 +0000 (0:00:00.064) 0:09:39.423 ******** 2026-08-18 14:51:53.205839 | node1 | ok: [node2] 2026-08-18 14:51:53.206062 | node1 | ok: [node3] 2026-08-18 14:51:53.206307 | node1 | 2026-08-18 14:51:53.206549 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:51:53.206787 | node1 | Tuesday 18 August 2026 14:51:53 +0000 (0:00:00.262) 0:09:39.685 ******** 2026-08-18 14:51:54.778418 | node1 | ok: [node3] 2026-08-18 14:51:54.778540 | node1 | ok: [node2] 2026-08-18 14:51:54.778552 | node1 | 2026-08-18 14:51:54.778562 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:51:54.778572 | node1 | Tuesday 18 August 2026 14:51:54 +0000 (0:00:01.571) 0:09:41.257 ******** 2026-08-18 14:51:54.844858 | node1 | ok: [node2] => { 2026-08-18 14:51:54.844999 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:51:54.845047 | node1 | } 2026-08-18 14:51:54.845251 | node1 | ok: [node3] => { 2026-08-18 14:51:54.845399 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:51:54.845529 | node1 | } 2026-08-18 14:51:54.845694 | node1 | 2026-08-18 14:51:54.845832 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:51:54.845962 | node1 | Tuesday 18 August 2026 14:51:54 +0000 (0:00:00.067) 0:09:41.324 ******** 2026-08-18 14:51:55.189054 | node1 | ok: [node2] 2026-08-18 14:51:55.189209 | node1 | ok: [node3] 2026-08-18 14:51:55.189244 | node1 | 2026-08-18 14:51:55.189260 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:51:55.189388 | node1 | Tuesday 18 August 2026 14:51:55 +0000 (0:00:00.344) 0:09:41.669 ******** 2026-08-18 14:51:57.209978 | node1 | ok: [node2] 2026-08-18 14:51:57.210026 | node1 | ok: [node3] 2026-08-18 14:51:57.210043 | node1 | 2026-08-18 14:51:57.210219 | node1 | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-08-18 14:51:57.210234 | node1 | Tuesday 18 August 2026 14:51:57 +0000 (0:00:02.021) 0:09:43.690 ******** 2026-08-18 14:51:57.254291 | node1 | skipping: [node2] 2026-08-18 14:51:57.254376 | node1 | skipping: [node3] 2026-08-18 14:51:57.254454 | node1 | 2026-08-18 14:51:57.254648 | node1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-08-18 14:51:57.254662 | node1 | Tuesday 18 August 2026 14:51:57 +0000 (0:00:00.044) 0:09:43.734 ******** 2026-08-18 14:51:57.295153 | node1 | skipping: [node2] 2026-08-18 14:51:57.295856 | node1 | skipping: [node3] 2026-08-18 14:51:57.295885 | node1 | 2026-08-18 14:51:57.295894 | node1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-08-18 14:51:57.295903 | node1 | Tuesday 18 August 2026 14:51:57 +0000 (0:00:00.040) 0:09:43.775 ******** 2026-08-18 14:51:57.342426 | node1 | skipping: [node2] 2026-08-18 14:51:57.342479 | node1 | skipping: [node3] 2026-08-18 14:51:57.342490 | node1 | 2026-08-18 14:51:57.342500 | node1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-08-18 14:51:57.342509 | node1 | Tuesday 18 August 2026 14:51:57 +0000 (0:00:00.045) 0:09:43.821 ******** 2026-08-18 14:51:58.655463 | node1 | ok: [node3] 2026-08-18 14:51:58.655509 | node1 | ok: [node2] 2026-08-18 14:51:58.655514 | node1 | 2026-08-18 14:51:58.655519 | node1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-08-18 14:51:58.655524 | node1 | Tuesday 18 August 2026 14:51:58 +0000 (0:00:01.314) 0:09:45.135 ******** 2026-08-18 14:51:59.123575 | node1 | ok: [node2] 2026-08-18 14:51:59.123664 | node1 | ok: [node3] 2026-08-18 14:51:59.123678 | node1 | 2026-08-18 14:51:59.123689 | node1 | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-08-18 14:51:59.123700 | node1 | Tuesday 18 August 2026 14:51:59 +0000 (0:00:00.467) 0:09:45.602 ******** 2026-08-18 14:52:00.017255 | node1 | ok: [node2] => (item={'path': '/etc/containerd'}) 2026-08-18 14:52:00.017315 | node1 | ok: [node3] => (item={'path': '/etc/containerd'}) 2026-08-18 14:52:00.017323 | node1 | ok: [node2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:52:00.017336 | node1 | ok: [node3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:52:00.017342 | node1 | ok: [node2] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:52:00.018108 | node1 | ok: [node3] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:52:00.018154 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:52:00.018160 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:52:00.018164 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:52:00.018169 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:52:00.018173 | node1 | 2026-08-18 14:52:00.018178 | node1 | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-08-18 14:52:00.018182 | node1 | Tuesday 18 August 2026 14:52:00 +0000 (0:00:00.894) 0:09:46.497 ******** 2026-08-18 14:52:00.555936 | node1 | ok: [node2] 2026-08-18 14:52:00.555978 | node1 | ok: [node3] 2026-08-18 14:52:00.555983 | node1 | 2026-08-18 14:52:00.555988 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:52:00.555992 | node1 | Tuesday 18 August 2026 14:52:00 +0000 (0:00:00.526) 0:09:47.023 ******** 2026-08-18 14:52:00.555996 | node1 | 2026-08-18 14:52:00.556000 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:52:00.556004 | node1 | Tuesday 18 August 2026 14:52:00 +0000 (0:00:00.006) 0:09:47.030 ******** 2026-08-18 14:52:00.556008 | node1 | 2026-08-18 14:52:00.556012 | node1 | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-08-18 14:52:00.556016 | node1 | Tuesday 18 August 2026 14:52:00 +0000 (0:00:00.005) 0:09:47.035 ******** 2026-08-18 14:52:00.937890 | node1 | ok: [node2] 2026-08-18 14:52:00.937962 | node1 | ok: [node3] 2026-08-18 14:52:00.938051 | node1 | 2026-08-18 14:52:00.938157 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:52:00.938313 | node1 | Tuesday 18 August 2026 14:52:00 +0000 (0:00:00.382) 0:09:47.417 ******** 2026-08-18 14:52:01.158867 | node1 | ok: [node2] 2026-08-18 14:52:01.158953 | node1 | ok: [node3] 2026-08-18 14:52:01.159158 | node1 | 2026-08-18 14:52:01.159229 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:52:01.159248 | node1 | Tuesday 18 August 2026 14:52:01 +0000 (0:00:00.221) 0:09:47.639 ******** 2026-08-18 14:52:01.227148 | node1 | ok: [node2] => { 2026-08-18 14:52:01.227270 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:52:01.228186 | node1 | } 2026-08-18 14:52:01.228231 | node1 | ok: [node3] => { 2026-08-18 14:52:01.228239 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:52:01.228245 | node1 | } 2026-08-18 14:52:01.228251 | node1 | 2026-08-18 14:52:01.228257 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:52:01.228327 | node1 | Tuesday 18 August 2026 14:52:01 +0000 (0:00:00.068) 0:09:47.707 ******** 2026-08-18 14:52:01.602007 | node1 | ok: [node2] 2026-08-18 14:52:01.602097 | node1 | ok: [node3] 2026-08-18 14:52:01.602505 | node1 | 2026-08-18 14:52:01.602554 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:52:01.602562 | node1 | Tuesday 18 August 2026 14:52:01 +0000 (0:00:00.375) 0:09:48.082 ******** 2026-08-18 14:52:04.769607 | node1 | ok: [node2] 2026-08-18 14:52:04.769681 | node1 | ok: [node3] 2026-08-18 14:52:04.769756 | node1 | 2026-08-18 14:52:04.769983 | node1 | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-08-18 14:52:04.770026 | node1 | Tuesday 18 August 2026 14:52:04 +0000 (0:00:03.167) 0:09:51.249 ******** 2026-08-18 14:52:06.137240 | node1 | ok: [node3] 2026-08-18 14:52:06.137309 | node1 | ok: [node2] 2026-08-18 14:52:06.137321 | node1 | 2026-08-18 14:52:06.137332 | node1 | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-08-18 14:52:06.137349 | node1 | Tuesday 18 August 2026 14:52:06 +0000 (0:00:01.366) 0:09:52.616 ******** 2026-08-18 14:52:06.380664 | node1 | ok: [node2] 2026-08-18 14:52:06.380789 | node1 | ok: [node3] 2026-08-18 14:52:06.380852 | node1 | 2026-08-18 14:52:06.381213 | node1 | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-08-18 14:52:06.381320 | node1 | Tuesday 18 August 2026 14:52:06 +0000 (0:00:00.243) 0:09:52.860 ******** 2026-08-18 14:52:06.852203 | node1 | ok: [node2] 2026-08-18 14:52:06.852287 | node1 | ok: [node3] 2026-08-18 14:52:06.852695 | node1 | 2026-08-18 14:52:06.852745 | node1 | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-08-18 14:52:06.852754 | node1 | Tuesday 18 August 2026 14:52:06 +0000 (0:00:00.471) 0:09:53.331 ******** 2026-08-18 14:52:07.418084 | node1 | ok: [node2] => (item={'path': '/etc/docker'}) 2026-08-18 14:52:07.418159 | node1 | ok: [node3] => (item={'path': '/etc/docker'}) 2026-08-18 14:52:07.418217 | node1 | ok: [node2] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:52:07.418802 | node1 | ok: [node3] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:52:07.418848 | node1 | ok: [node2] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:52:07.418853 | node1 | ok: [node3] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:52:07.418858 | node1 | 2026-08-18 14:52:07.418863 | node1 | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-08-18 14:52:07.418867 | node1 | Tuesday 18 August 2026 14:52:07 +0000 (0:00:00.566) 0:09:53.898 ******** 2026-08-18 14:52:07.869522 | node1 | ok: [node2] 2026-08-18 14:52:07.869595 | node1 | ok: [node3] 2026-08-18 14:52:07.870105 | node1 | 2026-08-18 14:52:07.870154 | node1 | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-08-18 14:52:07.870161 | node1 | Tuesday 18 August 2026 14:52:07 +0000 (0:00:00.451) 0:09:54.349 ******** 2026-08-18 14:52:08.311567 | node1 | ok: [node2] 2026-08-18 14:52:08.311635 | node1 | ok: [node3] 2026-08-18 14:52:08.311704 | node1 | 2026-08-18 14:52:08.312432 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:52:08.312476 | node1 | Tuesday 18 August 2026 14:52:08 +0000 (0:00:00.431) 0:09:54.780 ******** 2026-08-18 14:52:08.312482 | node1 | 2026-08-18 14:52:08.312486 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:52:08.312490 | node1 | Tuesday 18 August 2026 14:52:08 +0000 (0:00:00.006) 0:09:54.786 ******** 2026-08-18 14:52:08.312494 | node1 | 2026-08-18 14:52:08.312498 | node1 | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-08-18 14:52:08.312511 | node1 | Tuesday 18 August 2026 14:52:08 +0000 (0:00:00.005) 0:09:54.791 ******** 2026-08-18 14:52:08.676715 | node1 | ok: [node2] 2026-08-18 14:52:08.676767 | node1 | ok: [node3] 2026-08-18 14:52:08.676855 | node1 | 2026-08-18 14:52:08.677052 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:52:08.677068 | node1 | Tuesday 18 August 2026 14:52:08 +0000 (0:00:00.365) 0:09:55.157 ******** 2026-08-18 14:52:08.717941 | node1 | ok: [node2] => { 2026-08-18 14:52:08.718078 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:52:08.718978 | node1 | } 2026-08-18 14:52:08.718999 | node1 | ok: [node3] => { 2026-08-18 14:52:08.719006 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:52:08.719012 | node1 | } 2026-08-18 14:52:08.719018 | node1 | 2026-08-18 14:52:08.719024 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:52:08.719030 | node1 | Tuesday 18 August 2026 14:52:08 +0000 (0:00:00.040) 0:09:55.197 ******** 2026-08-18 14:52:09.001268 | node1 | ok: [node2] 2026-08-18 14:52:09.001380 | node1 | ok: [node3] 2026-08-18 14:52:09.001504 | node1 | 2026-08-18 14:52:09.001643 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:52:09.001781 | node1 | Tuesday 18 August 2026 14:52:09 +0000 (0:00:00.283) 0:09:55.481 ******** 2026-08-18 14:52:09.053330 | node1 | skipping: [node2] 2026-08-18 14:52:09.053519 | node1 | skipping: [node3] 2026-08-18 14:52:09.053709 | node1 | 2026-08-18 14:52:09.053922 | node1 | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-08-18 14:52:09.054155 | node1 | Tuesday 18 August 2026 14:52:09 +0000 (0:00:00.052) 0:09:55.533 ******** 2026-08-18 14:52:09.109540 | node1 | ok: [node2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/ubuntu-22.04.yml) 2026-08-18 14:52:09.109747 | node1 | ok: [node3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/ubuntu-22.04.yml) 2026-08-18 14:52:09.109940 | node1 | 2026-08-18 14:52:09.110205 | node1 | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-08-18 14:52:09.110425 | node1 | Tuesday 18 August 2026 14:52:09 +0000 (0:00:00.055) 0:09:55.589 ******** 2026-08-18 14:52:10.311680 | node1 | ok: [node3] 2026-08-18 14:52:10.311755 | node1 | ok: [node2] 2026-08-18 14:52:10.311768 | node1 | 2026-08-18 14:52:10.311874 | node1 | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-08-18 14:52:10.311968 | node1 | Tuesday 18 August 2026 14:52:10 +0000 (0:00:01.202) 0:09:56.791 ******** 2026-08-18 14:52:10.992465 | node1 | ok: [node2] => (item=chronyd) 2026-08-18 14:52:10.992675 | node1 | ok: [node3] => (item=chronyd) 2026-08-18 14:52:10.992983 | node1 | ok: [node2] => (item=sshd) 2026-08-18 14:52:10.993066 | node1 | ok: [node3] => (item=sshd) 2026-08-18 14:52:10.993296 | node1 | 2026-08-18 14:52:10.993499 | node1 | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-08-18 14:52:10.993699 | node1 | Tuesday 18 August 2026 14:52:10 +0000 (0:00:00.680) 0:09:57.472 ******** 2026-08-18 14:52:11.214612 | node1 | ok: [node2] 2026-08-18 14:52:11.214797 | node1 | ok: [node3] 2026-08-18 14:52:11.215016 | node1 | 2026-08-18 14:52:11.215220 | node1 | TASK [vexxhost.ceph.cephadm : Ensure the cephadm SSH user exists] ************** 2026-08-18 14:52:11.215422 | node1 | Tuesday 18 August 2026 14:52:11 +0000 (0:00:00.222) 0:09:57.694 ******** 2026-08-18 14:52:11.469788 | node1 | ok: [node2] 2026-08-18 14:52:11.469944 | node1 | ok: [node3] 2026-08-18 14:52:11.470183 | node1 | 2026-08-18 14:52:11.470408 | node1 | TASK [vexxhost.ceph.cephadm : Grant passwordless sudo to the cephadm SSH user] *** 2026-08-18 14:52:11.470660 | node1 | Tuesday 18 August 2026 14:52:11 +0000 (0:00:00.254) 0:09:57.949 ******** 2026-08-18 14:52:11.896984 | node1 | ok: [node2] 2026-08-18 14:52:11.897205 | node1 | ok: [node3] 2026-08-18 14:52:11.897393 | node1 | 2026-08-18 14:52:11.897606 | node1 | TASK [vexxhost.ceph.cephadm : Log in to the container registry] **************** 2026-08-18 14:52:11.897810 | node1 | Tuesday 18 August 2026 14:52:11 +0000 (0:00:00.427) 0:09:58.376 ******** 2026-08-18 14:52:11.952485 | node1 | skipping: [node2] 2026-08-18 14:52:11.952625 | node1 | skipping: [node3] 2026-08-18 14:52:11.952809 | node1 | 2026-08-18 14:52:11.953034 | node1 | TASK [vexxhost.ceph.cephadm : Pre-pull the Ceph image] ************************* 2026-08-18 14:52:11.953271 | node1 | Tuesday 18 August 2026 14:52:11 +0000 (0:00:00.055) 0:09:58.432 ******** 2026-08-18 14:52:12.341891 | node1 | ok: [node2] 2026-08-18 14:52:12.341973 | node1 | ok: [node3] 2026-08-18 14:52:12.342072 | node1 | 2026-08-18 14:52:12.342220 | node1 | TASK [vexxhost.ceph.cephadm : Pin CEPHADM_IMAGE for interactive login shells] *** 2026-08-18 14:52:12.342350 | node1 | Tuesday 18 August 2026 14:52:12 +0000 (0:00:00.389) 0:09:58.822 ******** 2026-08-18 14:52:12.778627 | node1 | ok: [node2] 2026-08-18 14:52:12.778687 | node1 | ok: [node3] 2026-08-18 14:52:12.778699 | node1 | 2026-08-18 14:52:12.778710 | node1 | TASK [vexxhost.ceph.monitoring : Locate the cluster admin host] **************** 2026-08-18 14:52:12.778720 | node1 | Tuesday 18 August 2026 14:52:12 +0000 (0:00:00.435) 0:09:59.257 ******** 2026-08-18 14:52:13.377146 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] => (item=node1) 2026-08-18 14:52:13.377208 | node1 | ok: [node2] => (item=node2) 2026-08-18 14:52:13.377602 | node1 | ok: [node2 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 14:52:13.377651 | node1 | 2026-08-18 14:52:13.377657 | node1 | TASK [vexxhost.ceph.monitoring : Select the admin host] ************************ 2026-08-18 14:52:13.377662 | node1 | Tuesday 18 August 2026 14:52:13 +0000 (0:00:00.599) 0:09:59.857 ******** 2026-08-18 14:52:13.441978 | node1 | ok: [node2] 2026-08-18 14:52:13.442079 | node1 | ok: [node3] 2026-08-18 14:52:13.442389 | node1 | 2026-08-18 14:52:13.442434 | node1 | TASK [vexxhost.ceph.monitoring : Set the admin mon config path] **************** 2026-08-18 14:52:13.442440 | node1 | Tuesday 18 August 2026 14:52:13 +0000 (0:00:00.064) 0:09:59.922 ******** 2026-08-18 14:52:13.508894 | node1 | ok: [node2] 2026-08-18 14:52:13.508976 | node1 | ok: [node3] 2026-08-18 14:52:13.509059 | node1 | 2026-08-18 14:52:13.509295 | node1 | TASK [vexxhost.ceph.monitoring : Pin monitoring container images] ************** 2026-08-18 14:52:13.509331 | node1 | Tuesday 18 August 2026 14:52:13 +0000 (0:00:00.066) 0:09:59.988 ******** 2026-08-18 14:52:13.544910 | node1 | skipping: [node2] 2026-08-18 14:52:13.544977 | node1 | 2026-08-18 14:52:13.545085 | node1 | TASK [vexxhost.ceph.monitoring : Apply the monitoring service specs] *********** 2026-08-18 14:52:13.545223 | node1 | Tuesday 18 August 2026 14:52:13 +0000 (0:00:00.036) 0:10:00.025 ******** 2026-08-18 14:52:16.288152 | node1 | changed: [node2 -> node1({{ nodepool.private_ipv4 }})] => (item=ceph-exporter) 2026-08-18 14:52:16.288221 | node1 | 2026-08-18 14:52:16.288306 | node1 | PLAY RECAP ********************************************************************* 2026-08-18 14:52:16.288417 | node1 | node1 : ok=204 changed=47 unreachable=0 failed=1 skipped=37 rescued=0 ignored=0 2026-08-18 14:52:16.288547 | node1 | node2 : ok=210 changed=29 unreachable=0 failed=0 skipped=53 rescued=0 ignored=0 2026-08-18 14:52:16.288646 | node1 | node3 : ok=206 changed=28 unreachable=0 failed=0 skipped=45 rescued=0 ignored=0 2026-08-18 14:52:16.288723 | node1 | 2026-08-18 14:52:16.288806 | node1 | 2026-08-18 14:52:16.288906 | node1 | TASKS RECAP ******************************************************************** 2026-08-18 14:52:16.289000 | node1 | Tuesday 18 August 2026 14:52:16 +0000 (0:00:02.743) 0:10:02.768 ******** 2026-08-18 14:52:16.289086 | node1 | =============================================================================== 2026-08-18 14:52:16.289205 | node1 | vexxhost.ceph.osd : Deploy OSDs on the explicitly listed devices ------ 222.16s 2026-08-18 14:52:16.289297 | node1 | vexxhost.ceph.cephadm : Pre-pull the Ceph image ------------------------ 41.86s 2026-08-18 14:52:16.289389 | node1 | vexxhost.ceph.mon : Bootstrap the cluster ------------------------------ 41.72s 2026-08-18 14:52:16.289481 | node1 | vexxhost.ceph.configure : Configure pools ------------------------------ 30.76s 2026-08-18 14:52:16.289575 | node1 | vexxhost.ceph.cephadm_host : Register the host (labels) ---------------- 26.70s 2026-08-18 14:52:16.289667 | node1 | vexxhost.ceph.configure : Apply central Ceph configuration ------------- 16.98s 2026-08-18 14:52:16.289767 | node1 | vexxhost.ceph.osd : Wait for explicitly requested OSD daemons to be running on this host -- 14.52s 2026-08-18 14:52:16.289864 | node1 | vexxhost.ceph.mgr : Configure ceph-mgr modules -------------------------- 9.73s 2026-08-18 14:52:16.289957 | node1 | vexxhost.ceph.configure : Configure CRUSH rules ------------------------- 9.15s 2026-08-18 14:52:16.290049 | node1 | vexxhost.ceph.configure : Configure dashboard users --------------------- 8.65s 2026-08-18 14:52:16.290145 | node1 | vexxhost.ceph.cephadm_host : Get the cephadm public key ----------------- 7.55s 2026-08-18 14:52:16.290242 | node1 | vexxhost.ceph.configure : Reload the dashboard so it picks up the certificate --- 6.70s 2026-08-18 14:52:16.290335 | node1 | vexxhost.ceph.cephadm : Install packages -------------------------------- 5.31s 2026-08-18 14:52:16.290428 | node1 | vexxhost.containers.containerd : Install AppArmor packages -------------- 5.26s 2026-08-18 14:52:16.290524 | node1 | vexxhost.containers.download_artifact : Extract archive ----------------- 4.45s 2026-08-18 14:52:16.290619 | node1 | vexxhost.containers.download_artifact : Extract archive ----------------- 4.07s 2026-08-18 14:52:16.290796 | node1 | vexxhost.ceph.rgw : Configure RGW realms -------------------------------- 3.96s 2026-08-18 14:52:16.290882 | node1 | vexxhost.ceph.configure : Configure erasure-code profiles --------------- 3.95s 2026-08-18 14:52:16.290978 | node1 | vexxhost.containers.download_artifact : Extract archive ----------------- 3.17s 2026-08-18 14:52:16.291123 | node1 | vexxhost.containers.download_artifact : Extract archive ----------------- 3.14s 2026-08-18 14:52:16.389653 | node1 | CRITICAL Ansible return code was 2, command was: ansible-playbook --inventory /home/zuul/.ansible/tmp/molecule.kre9.default/inventory --skip-tags molecule-notest,notest --inventory=/home/zuul/src/github.com/vexxhost/ansible-collection-ceph/inventory.yaml /home/zuul/src/github.com/vexxhost/ansible-collection-ceph/extensions/molecule/default/converge.yml 2026-08-18 14:52:16.389696 | node1 | ERROR [default > converge] Executed: Failed 2026-08-18 14:52:16.389787 | node1 | ERROR Ansible return code was 2, command was: ansible-playbook --inventory /home/zuul/.ansible/tmp/molecule.kre9.default/inventory --skip-tags molecule-notest,notest --inventory=/home/zuul/src/github.com/vexxhost/ansible-collection-ceph/inventory.yaml /home/zuul/src/github.com/vexxhost/ansible-collection-ceph/extensions/molecule/default/converge.yml 2026-08-18 14:52:26.941709 | node1 | ERROR 2026-08-18 14:52:26.941929 | node1 | { 2026-08-18 14:52:26.941972 | node1 | "delta": "0:11:16.007219", 2026-08-18 14:52:26.942001 | node1 | "end": "2026-08-18 14:52:16.451579", 2026-08-18 14:52:26.942028 | node1 | "msg": "non-zero return code", 2026-08-18 14:52:26.942070 | node1 | "rc": 2, 2026-08-18 14:52:26.942096 | node1 | "start": "2026-08-18 14:41:00.444360" 2026-08-18 14:52:26.942120 | node1 | } failure 2026-08-18 14:52:26.964844 | 2026-08-18 14:52:26.964902 | PLAY RECAP 2026-08-18 14:52:26.964949 | node1 | ok: 3 changed: 2 unreachable: 0 failed: 1 skipped: 0 rescued: 0 ignored: 0 2026-08-18 14:52:26.964990 | 2026-08-18 14:52:27.108325 | RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/run.yaml@main] 2026-08-18 14:52:27.125623 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main] 2026-08-18 14:52:27.757167 | 2026-08-18 14:52:27.757284 | PLAY [all] 2026-08-18 14:52:27.769147 | 2026-08-18 14:52:27.769226 | TASK [fetch-output : Set log path for multiple nodes] 2026-08-18 14:52:27.819905 | node1 | ok 2026-08-18 14:52:27.825789 | node2 | ok 2026-08-18 14:52:27.830257 | node3 | ok 2026-08-18 14:52:27.835027 | 2026-08-18 14:52:27.835114 | TASK [fetch-output : Set log path for single node] 2026-08-18 14:52:27.866925 | node1 | skipping: Conditional result was False 2026-08-18 14:52:27.878235 | node2 | skipping: Conditional result was False 2026-08-18 14:52:27.878673 | node3 | skipping: Conditional result was False 2026-08-18 14:52:27.887059 | 2026-08-18 14:52:27.887148 | LOOP [fetch-output : Ensure local output dirs] 2026-08-18 14:52:28.262466 | node1 -> localhost | changed: "/var/lib/zuul/builds/80089b70c9bf4e9cbfe604215db5f29f/work/logs/node1" 2026-08-18 14:52:28.272980 | node2 -> localhost | changed: "/var/lib/zuul/builds/80089b70c9bf4e9cbfe604215db5f29f/work/logs/node2" 2026-08-18 14:52:28.280253 | node3 -> localhost | changed: "/var/lib/zuul/builds/80089b70c9bf4e9cbfe604215db5f29f/work/logs/node3" 2026-08-18 14:52:28.477984 | node1 -> localhost | changed: "/var/lib/zuul/builds/80089b70c9bf4e9cbfe604215db5f29f/work/artifacts" 2026-08-18 14:52:28.497065 | node2 -> localhost | ok: "/var/lib/zuul/builds/80089b70c9bf4e9cbfe604215db5f29f/work/artifacts" 2026-08-18 14:52:28.561844 | node3 -> localhost | ok: "/var/lib/zuul/builds/80089b70c9bf4e9cbfe604215db5f29f/work/artifacts" 2026-08-18 14:52:28.688334 | node1 -> localhost | changed: "/var/lib/zuul/builds/80089b70c9bf4e9cbfe604215db5f29f/work/docs" 2026-08-18 14:52:28.716223 | node2 -> localhost | ok: "/var/lib/zuul/builds/80089b70c9bf4e9cbfe604215db5f29f/work/docs" 2026-08-18 14:52:28.716430 | node2 -> localhost | changed: All items complete 2026-08-18 14:52:28.716486 | 2026-08-18 14:52:28.770290 | node3 -> localhost | ok: "/var/lib/zuul/builds/80089b70c9bf4e9cbfe604215db5f29f/work/docs" 2026-08-18 14:52:28.825420 | 2026-08-18 14:52:28.825550 | LOOP [fetch-output : Collect logs, artifacts and docs] 2026-08-18 14:52:29.415815 | node3 | changed: .d..t...... ./ 2026-08-18 14:52:29.416067 | node3 | changed: All items complete 2026-08-18 14:52:29.416107 | 2026-08-18 14:52:29.422981 | node1 | changed: .d..t...... ./ 2026-08-18 14:52:29.430987 | node2 | changed: .d..t...... ./ 2026-08-18 14:52:29.868069 | node1 | changed: .d..t...... ./ 2026-08-18 14:52:29.881947 | node2 | ok 2026-08-18 14:52:29.887666 | node3 | ok 2026-08-18 14:52:30.303938 | node1 | changed: .d..t...... ./ 2026-08-18 14:52:30.317086 | node3 | ok 2026-08-18 14:52:30.338050 | node2 | ok 2026-08-18 14:52:30.338171 | node2 | changed: All items complete 2026-08-18 14:52:30.338232 | 2026-08-18 14:52:30.370663 | 2026-08-18 14:52:30.371000 | LOOP [merge-output-to-logs : Move artifacts and docs to logs dir] 2026-08-18 14:52:30.730709 | node1 -> localhost | ok: Item: artifacts Runtime: 0:00:00.007766 2026-08-18 14:52:30.943569 | node1 -> localhost | ok: Item: docs Runtime: 0:00:00.008952 2026-08-18 14:52:30.975649 | 2026-08-18 14:52:30.975757 | PLAY [all] 2026-08-18 14:52:30.982267 | 2026-08-18 14:52:30.982330 | TASK [remove-build-sshkey : Remove the build SSH key from all nodes] 2026-08-18 14:52:31.397956 | node1 | changed 2026-08-18 14:52:31.555643 | node3 | changed 2026-08-18 14:52:31.604727 | node2 | changed 2026-08-18 14:52:31.620041 | 2026-08-18 14:52:31.620098 | PLAY RECAP 2026-08-18 14:52:31.620143 | node1 | ok: 5 changed: 4 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0 2026-08-18 14:52:31.620171 | node2 | ok: 4 changed: 3 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0 2026-08-18 14:52:31.620195 | node3 | ok: 4 changed: 3 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0 2026-08-18 14:52:31.620213 | 2026-08-18 14:52:31.734308 | POST-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main] 2026-08-18 14:52:31.750454 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post-logs.yaml@main] 2026-08-18 14:52:32.358956 | 2026-08-18 14:52:32.359501 | PLAY [localhost] 2026-08-18 14:52:32.369412 | 2026-08-18 14:52:32.369507 | TASK [Generate Zuul manifest] 2026-08-18 14:52:32.391623 | localhost | ok 2026-08-18 14:52:32.415838 | 2026-08-18 14:52:32.416032 | TASK [generate-zuul-manifest : Generate Zuul manifest] 2026-08-18 14:52:32.770710 | localhost | changed 2026-08-18 14:52:32.785225 | 2026-08-18 14:52:32.785345 | TASK [generate-zuul-manifest : Return Zuul manifest URL to Zuul] 2026-08-18 14:52:32.816704 | localhost | ok 2026-08-18 14:52:32.825446 | 2026-08-18 14:52:32.825516 | TASK [Upload logs] 2026-08-18 14:52:32.847184 | localhost | ok 2026-08-18 14:52:32.941966 | 2026-08-18 14:52:32.942095 | TASK [Set zuul-log-path fact] 2026-08-18 14:52:32.964036 | localhost | ok 2026-08-18 14:52:32.977538 | 2026-08-18 14:52:32.977607 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-08-18 14:52:33.007105 | localhost | ok 2026-08-18 14:52:33.016850 | 2026-08-18 14:52:33.017017 | TASK [upload-logs : Create log directories] 2026-08-18 14:52:33.361755 | localhost | changed 2026-08-18 14:52:33.368037 | 2026-08-18 14:52:33.368107 | TASK [upload-logs : Ensure logs are readable before uploading] 2026-08-18 14:52:33.743528 | localhost -> localhost | ok: Runtime: 0:00:00.005517 2026-08-18 14:52:33.750575 | 2026-08-18 14:52:33.750669 | TASK [upload-logs : Upload logs to log server] 2026-08-18 14:52:34.219719 | localhost | Output suppressed because no_log was given 2026-08-18 14:52:34.225832 | 2026-08-18 14:52:34.225922 | LOOP [upload-logs : Compress console log and json output] 2026-08-18 14:52:34.269913 | localhost | skipping: Conditional result was False 2026-08-18 14:52:34.276604 | localhost | skipping: Conditional result was False 2026-08-18 14:52:34.289551 | 2026-08-18 14:52:34.289758 | LOOP [upload-logs : Upload compressed console log and json output] 2026-08-18 14:52:34.334913 | localhost | skipping: Conditional result was False 2026-08-18 14:52:34.335448 | 2026-08-18 14:52:34.338031 | localhost | skipping: Conditional result was False 2026-08-18 14:52:34.356726 | 2026-08-18 14:52:34.356925 | LOOP [upload-logs : Upload console log and json output]