2026-02-25 16:36:14.813150 | Job console starting 2026-02-25 16:36:14.824054 | Updating git repos 2026-02-25 16:36:14.888101 | Cloning repos into workspace 2026-02-25 16:36:15.034492 | Restoring repo states 2026-02-25 16:36:15.055755 | Merging changes 2026-02-25 16:36:16.365320 | Checking out repos 2026-02-25 16:36:16.544697 | Preparing playbooks 2026-02-25 16:36:19.473767 | Running Ansible setup 2026-02-25 16:36:22.854456 | PRE-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main] 2026-02-25 16:36:23.491375 | 2026-02-25 16:36:23.491590 | PLAY [localhost] 2026-02-25 16:36:23.502079 | 2026-02-25 16:36:23.502189 | TASK [Gathering Facts] 2026-02-25 16:36:24.388096 | localhost | ok 2026-02-25 16:36:24.399716 | 2026-02-25 16:36:24.399883 | TASK [Setup log path fact] 2026-02-25 16:36:24.442479 | localhost | ok 2026-02-25 16:36:24.456930 | 2026-02-25 16:36:24.457085 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-02-25 16:36:24.491060 | localhost | ok 2026-02-25 16:36:24.498308 | 2026-02-25 16:36:24.498423 | TASK [emit-job-header : Print job information] 2026-02-25 16:36:24.540226 | # Job Information 2026-02-25 16:36:24.540465 | Ansible Version: 2.16.16 2026-02-25 16:36:24.540510 | Job: atmosphere-molecule-csi-rbd 2026-02-25 16:36:24.540542 | Pipeline: check 2026-02-25 16:36:24.540572 | Executor: 0a8996d2b663 2026-02-25 16:36:24.540601 | Triggered by: https://github.com/vexxhost/atmosphere/pull/3443 2026-02-25 16:36:24.540635 | Event ID: 03ee3e80-1268-11f1-8e4b-7fff7fd8dc05 2026-02-25 16:36:24.544851 | 2026-02-25 16:36:24.544948 | LOOP [emit-job-header : Print node information] 2026-02-25 16:36:24.631070 | localhost | ok: 2026-02-25 16:36:24.631335 | localhost | # Node Information 2026-02-25 16:36:24.631386 | localhost | Inventory Hostname: instance 2026-02-25 16:36:24.631423 | localhost | Hostname: np0000156984 2026-02-25 16:36:24.631458 | localhost | Username: zuul 2026-02-25 16:36:24.631499 | localhost | Distro: Ubuntu 22.04 2026-02-25 16:36:24.631533 | localhost | Provider: yul1 2026-02-25 16:36:24.631566 | localhost | Region: ca-ymq-1 2026-02-25 16:36:24.631599 | localhost | Label: ubuntu-jammy 2026-02-25 16:36:24.631652 | localhost | Product Name: OpenStack Nova 2026-02-25 16:36:24.631683 | localhost | Interface IP: 199.204.45.162 2026-02-25 16:36:24.643321 | 2026-02-25 16:36:24.643503 | TASK [log-inventory : Ensure Zuul Ansible directory exists] 2026-02-25 16:36:25.067267 | localhost -> localhost | changed 2026-02-25 16:36:25.077048 | 2026-02-25 16:36:25.077160 | TASK [log-inventory : Copy ansible inventory to logs dir] 2026-02-25 16:36:25.926536 | localhost -> localhost | changed 2026-02-25 16:36:25.936385 | 2026-02-25 16:36:25.936519 | PLAY [all] 2026-02-25 16:36:25.947246 | 2026-02-25 16:36:25.947375 | TASK [add-build-sshkey : Check to see if ssh key was already created for this build] 2026-02-25 16:36:26.195569 | instance -> localhost | ok 2026-02-25 16:36:26.205304 | 2026-02-25 16:36:26.205396 | TASK [add-build-sshkey : Create a new key in workspace based on build UUID] 2026-02-25 16:36:26.237966 | instance | ok 2026-02-25 16:36:26.252444 | instance | included: /var/lib/zuul/builds/b047b56ca6f64c5ca72689c14df52264/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml 2026-02-25 16:36:26.258328 | 2026-02-25 16:36:26.258399 | TASK [add-build-sshkey : Create Temp SSH key] 2026-02-25 16:36:27.240926 | instance -> localhost | Generating public/private rsa key pair. 2026-02-25 16:36:27.241171 | instance -> localhost | Your identification has been saved in /var/lib/zuul/builds/b047b56ca6f64c5ca72689c14df52264/work/b047b56ca6f64c5ca72689c14df52264_id_rsa 2026-02-25 16:36:27.241213 | instance -> localhost | Your public key has been saved in /var/lib/zuul/builds/b047b56ca6f64c5ca72689c14df52264/work/b047b56ca6f64c5ca72689c14df52264_id_rsa.pub 2026-02-25 16:36:27.241245 | instance -> localhost | The key fingerprint is: 2026-02-25 16:36:27.241275 | instance -> localhost | SHA256:ABwN2UBG+Qg0hNq2cA/FZwaqIMwjNKyl7RSB1w/RW6c zuul-build-sshkey 2026-02-25 16:36:27.241318 | instance -> localhost | The key's randomart image is: 2026-02-25 16:36:27.241348 | instance -> localhost | +---[RSA 3072]----+ 2026-02-25 16:36:27.241386 | instance -> localhost | |.B=B@% | 2026-02-25 16:36:27.241416 | instance -> localhost | |*.*+O.B . . | 2026-02-25 16:36:27.241445 | instance -> localhost | |*X.+ O.o o | 2026-02-25 16:36:27.241473 | instance -> localhost | |O.O . +.E | 2026-02-25 16:36:27.241501 | instance -> localhost | |.* + S | 2026-02-25 16:36:27.241528 | instance -> localhost | | o . | 2026-02-25 16:36:27.241555 | instance -> localhost | | | 2026-02-25 16:36:27.241583 | instance -> localhost | | | 2026-02-25 16:36:27.241614 | instance -> localhost | | | 2026-02-25 16:36:27.241642 | instance -> localhost | +----[SHA256]-----+ 2026-02-25 16:36:27.241704 | instance -> localhost | ok: Runtime: 0:00:00.559401 2026-02-25 16:36:27.247043 | 2026-02-25 16:36:27.247107 | TASK [add-build-sshkey : Remote setup ssh keys (linux)] 2026-02-25 16:36:27.279222 | instance | ok 2026-02-25 16:36:27.288898 | instance | included: /var/lib/zuul/builds/b047b56ca6f64c5ca72689c14df52264/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml 2026-02-25 16:36:27.296155 | 2026-02-25 16:36:27.296213 | TASK [add-build-sshkey : Remove previously added zuul-build-sshkey] 2026-02-25 16:36:27.320569 | instance | skipping: Conditional result was False 2026-02-25 16:36:27.330986 | 2026-02-25 16:36:27.331075 | TASK [add-build-sshkey : Enable access via build key on all nodes] 2026-02-25 16:36:27.783754 | instance | changed 2026-02-25 16:36:27.789605 | 2026-02-25 16:36:27.789670 | TASK [add-build-sshkey : Make sure user has a .ssh] 2026-02-25 16:36:27.974919 | instance | ok 2026-02-25 16:36:27.980645 | 2026-02-25 16:36:27.980724 | TASK [add-build-sshkey : Install build private key as SSH key on all nodes] 2026-02-25 16:36:28.473449 | instance | changed 2026-02-25 16:36:28.479643 | 2026-02-25 16:36:28.479794 | TASK [add-build-sshkey : Install build public key as SSH key on all nodes] 2026-02-25 16:36:28.933109 | instance | changed 2026-02-25 16:36:28.938981 | 2026-02-25 16:36:28.939052 | TASK [add-build-sshkey : Remote setup ssh keys (windows)] 2026-02-25 16:36:28.963914 | instance | skipping: Conditional result was False 2026-02-25 16:36:28.973001 | 2026-02-25 16:36:28.973116 | TASK [remove-zuul-sshkey : Remove master key from local agent] 2026-02-25 16:36:29.394404 | instance -> localhost | changed 2026-02-25 16:36:29.407564 | 2026-02-25 16:36:29.407628 | TASK [add-build-sshkey : Add back temp key] 2026-02-25 16:36:29.707444 | instance -> localhost | Identity added: /var/lib/zuul/builds/b047b56ca6f64c5ca72689c14df52264/work/b047b56ca6f64c5ca72689c14df52264_id_rsa (zuul-build-sshkey) 2026-02-25 16:36:29.707649 | instance -> localhost | ok: Runtime: 0:00:00.014199 2026-02-25 16:36:29.714334 | 2026-02-25 16:36:29.714456 | TASK [add-build-sshkey : Verify we can still SSH to all nodes] 2026-02-25 16:36:30.049345 | instance | ok 2026-02-25 16:36:30.053927 | 2026-02-25 16:36:30.053991 | TASK [add-build-sshkey : Verify we can still SSH to all nodes (windows)] 2026-02-25 16:36:30.077024 | instance | skipping: Conditional result was False 2026-02-25 16:36:30.090422 | 2026-02-25 16:36:30.090482 | TASK [prepare-workspace : Start zuul_console daemon.] 2026-02-25 16:36:30.415969 | instance | ok 2026-02-25 16:36:30.422958 | 2026-02-25 16:36:30.423022 | TASK [prepare-workspace : Synchronize src repos to workspace directory.] 2026-02-25 16:36:32.148599 | instance | Output suppressed because no_log was given 2026-02-25 16:36:32.159212 | 2026-02-25 16:36:32.159283 | LOOP [ensure-output-dirs : Empty Zuul Output directories by removing them] 2026-02-25 16:36:32.335694 | instance | ok: "logs" 2026-02-25 16:36:32.336343 | instance | ok: All items complete 2026-02-25 16:36:32.336391 | 2026-02-25 16:36:32.479811 | instance | ok: "artifacts" 2026-02-25 16:36:32.630546 | instance | ok: "docs" 2026-02-25 16:36:32.640752 | 2026-02-25 16:36:32.640827 | LOOP [ensure-output-dirs : Ensure Zuul Output directories exist] 2026-02-25 16:36:32.835176 | instance | changed: "logs" 2026-02-25 16:36:32.982362 | instance | changed: "artifacts" 2026-02-25 16:36:33.131342 | instance | changed: "docs" 2026-02-25 16:36:33.150555 | 2026-02-25 16:36:33.150675 | PLAY RECAP 2026-02-25 16:36:33.150727 | instance | ok: 15 changed: 8 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0 2026-02-25 16:36:33.150756 | localhost | ok: 6 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-02-25 16:36:33.150776 | 2026-02-25 16:36:33.262990 | PRE-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main] 2026-02-25 16:36:33.266891 | PRE-RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main] 2026-02-25 16:36:33.893536 | 2026-02-25 16:36:33.893741 | PLAY [all] 2026-02-25 16:36:33.908499 | 2026-02-25 16:36:33.908728 | TASK [setup-uv : Extract archive] 2026-02-25 16:36:36.076524 | instance | changed 2026-02-25 16:36:36.083447 | 2026-02-25 16:36:36.083543 | TASK [setup-uv : Print version] 2026-02-25 16:36:35.647044 | instance | uv 0.8.13 2026-02-25 16:36:36.618692 | instance | ok: Runtime: 0:00:00.012116 2026-02-25 16:36:36.626272 | 2026-02-25 16:36:36.626386 | PLAY RECAP 2026-02-25 16:36:36.626433 | instance | ok: 2 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-02-25 16:36:36.626457 | 2026-02-25 16:36:36.779978 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main] 2026-02-25 16:36:36.782190 | PRE-RUN START: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/pre.yml@main] 2026-02-25 16:36:37.398109 | 2026-02-25 16:36:37.398317 | PLAY [all] 2026-02-25 16:36:37.410743 | 2026-02-25 16:36:37.410862 | TASK [Install "jq" for log collection] 2026-02-25 16:36:46.878079 | instance | changed 2026-02-25 16:36:46.880320 | 2026-02-25 16:36:46.880408 | PLAY RECAP 2026-02-25 16:36:46.880494 | instance | ok: 1 changed: 1 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-02-25 16:36:46.880583 | 2026-02-25 16:36:47.056854 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/pre.yml@main] 2026-02-25 16:36:47.060797 | RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/run.yaml@main] 2026-02-25 16:36:47.661245 | 2026-02-25 16:36:47.661388 | PLAY [all] 2026-02-25 16:36:47.673587 | 2026-02-25 16:36:47.673731 | TASK [Copy inventory file for Zuul] 2026-02-25 16:36:48.503448 | instance | changed 2026-02-25 16:36:48.510739 | 2026-02-25 16:36:48.510852 | TASK [Switch "ansible_host" to private IP] 2026-02-25 16:36:48.801455 | instance | changed: 1 replacements made 2026-02-25 16:36:48.808123 | 2026-02-25 16:36:48.808231 | TASK [Run Molecule scenario] 2026-02-25 16:36:49.199305 | instance | Using CPython 3.10.12 interpreter at: /usr/bin/python3 2026-02-25 16:36:49.199495 | instance | Creating virtual environment at: .venv 2026-02-25 16:36:49.222348 | instance | Building atmosphere @ file:///home/zuul/src/github.com/vexxhost/atmosphere 2026-02-25 16:36:49.241484 | instance | Downloading pygments (1.2MiB) 2026-02-25 16:36:49.255763 | instance | Downloading pydantic-core (2.0MiB) 2026-02-25 16:36:49.256473 | instance | Downloading netaddr (2.2MiB) 2026-02-25 16:36:49.257233 | instance | Downloading openstacksdk (1.7MiB) 2026-02-25 16:36:49.257461 | instance | Downloading cryptography (4.2MiB) 2026-02-25 16:36:49.257966 | instance | Downloading rjsonnet (1.2MiB) 2026-02-25 16:36:49.258214 | instance | Downloading ansible-core (2.1MiB) 2026-02-25 16:36:49.258607 | instance | Downloading kubernetes (1.9MiB) 2026-02-25 16:36:49.264501 | instance | Downloading setuptools (1.1MiB) 2026-02-25 16:36:49.531715 | instance | Building pyperclip==1.9.0 2026-02-25 16:36:49.540359 | instance | Downloading rjsonnet 2026-02-25 16:36:49.644034 | instance | Downloading pydantic-core 2026-02-25 16:36:49.690523 | instance | Downloading netaddr 2026-02-25 16:36:49.705098 | instance | Downloading pygments 2026-02-25 16:36:49.717848 | instance | Downloading cryptography 2026-02-25 16:36:49.746738 | instance | Downloading setuptools 2026-02-25 16:36:49.795595 | instance | Downloading kubernetes 2026-02-25 16:36:49.825196 | instance | Downloading ansible-core 2026-02-25 16:36:49.856088 | instance | Downloading openstacksdk 2026-02-25 16:36:50.231150 | instance | Built pyperclip==1.9.0 2026-02-25 16:36:50.433480 | instance | Built atmosphere @ file:///home/zuul/src/github.com/vexxhost/atmosphere 2026-02-25 16:36:50.466978 | instance | Installed 83 packages in 31ms 2026-02-25 16:36:51.086335 | instance | WARNING Molecule scenarios should migrate to 'extensions/molecule' 2026-02-25 16:36:51.699028 | instance | INFO [csi > discovery] scenario test matrix: dependency, cleanup, destroy, syntax, create, prepare, converge, idempotence, side_effect, verify, cleanup, destroy 2026-02-25 16:36:51.699097 | instance | INFO [csi > prerun] Performing prerun with role_name_check=0... 2026-02-25 16:37:31.457594 | instance | INFO [csi > dependency] Executing 2026-02-25 16:37:31.457802 | instance | WARNING [csi > dependency] Missing roles requirements file: requirements.yml 2026-02-25 16:37:31.458082 | instance | WARNING [csi > dependency] Missing collections requirements file: collections.yml 2026-02-25 16:37:31.458241 | instance | WARNING [csi > dependency] Executed: 2 missing (Remove from test_sequence to suppress) 2026-02-25 16:37:31.466255 | instance | INFO [csi > cleanup] Executing 2026-02-25 16:37:31.466584 | instance | WARNING [csi > cleanup] Executed: Missing playbook (Remove from test_sequence to suppress) 2026-02-25 16:37:31.474072 | instance | INFO [csi > destroy] Executing 2026-02-25 16:37:31.474109 | instance | WARNING [csi > destroy] Skipping, '--destroy=never' requested. 2026-02-25 16:37:31.474223 | instance | INFO [csi > destroy] Executed: Successful 2026-02-25 16:37:31.482319 | instance | INFO [csi > syntax] Executing 2026-02-25 16:37:32.969845 | instance | 2026-02-25 16:37:32.970202 | instance | playbook: /home/zuul/src/github.com/vexxhost/atmosphere/molecule/csi/converge.yml 2026-02-25 16:37:33.024810 | instance | INFO [csi > syntax] Executed: Successful 2026-02-25 16:37:33.037345 | instance | INFO [csi > create] Executing 2026-02-25 16:37:33.039368 | instance | WARNING [csi > create] Executed: Missing playbook (Remove from test_sequence to suppress) 2026-02-25 16:37:33.046935 | instance | INFO [csi > prepare] Executing 2026-02-25 16:37:33.820581 | instance | 2026-02-25 16:37:33.820919 | instance | PLAY [Prepare] ***************************************************************** 2026-02-25 16:37:33.821224 | instance | 2026-02-25 16:37:33.821613 | instance | TASK [Gathering Facts] ********************************************************* 2026-02-25 16:37:33.821928 | instance | Wednesday 25 February 2026 16:37:33 +0000 (0:00:00.027) 0:00:00.027 **** 2026-02-25 16:37:34.940094 | instance | [WARNING]: Platform linux on host instance is using the discovered Python 2026-02-25 16:37:34.940355 | instance | interpreter at /usr/bin/python3.10, but future installation of another Python 2026-02-25 16:37:34.940626 | instance | interpreter could change the meaning of that path. See 2026-02-25 16:37:34.940897 | instance | https://docs.ansible.com/ansible- 2026-02-25 16:37:34.941170 | instance | core/2.17/reference_appendices/interpreter_discovery.html for more information. 2026-02-25 16:37:34.948602 | instance | ok: [instance] 2026-02-25 16:37:34.948754 | instance | 2026-02-25 16:37:34.948922 | instance | TASK [Configure short hostname] ************************************************ 2026-02-25 16:37:34.949097 | instance | Wednesday 25 February 2026 16:37:34 +0000 (0:00:01.128) 0:00:01.156 **** 2026-02-25 16:37:35.584150 | instance | changed: [instance] 2026-02-25 16:37:35.584390 | instance | 2026-02-25 16:37:35.584663 | instance | TASK [Ensure hostname inside hosts file] *************************************** 2026-02-25 16:37:35.584936 | instance | Wednesday 25 February 2026 16:37:35 +0000 (0:00:00.635) 0:00:01.791 **** 2026-02-25 16:37:35.835792 | instance | [WARNING]: Module remote_tmp /root/.ansible/tmp did not exist and was created 2026-02-25 16:37:35.836030 | instance | with a mode of 0700, this may cause issues when running as another user. To 2026-02-25 16:37:35.836342 | instance | avoid this, create the remote_tmp dir with the correct permissions manually 2026-02-25 16:37:35.846866 | instance | changed: [instance] 2026-02-25 16:37:35.847024 | instance | 2026-02-25 16:37:35.847365 | instance | TASK [Purge "snapd" package] *************************************************** 2026-02-25 16:37:35.847576 | instance | Wednesday 25 February 2026 16:37:35 +0000 (0:00:00.262) 0:00:02.054 **** 2026-02-25 16:37:36.585582 | instance | ok: [instance] 2026-02-25 16:37:36.585809 | instance | 2026-02-25 16:37:36.586127 | instance | PLAY [Create devices for Ceph] ************************************************* 2026-02-25 16:37:36.586503 | instance | 2026-02-25 16:37:36.586860 | instance | TASK [Gathering Facts] ********************************************************* 2026-02-25 16:37:36.587160 | instance | Wednesday 25 February 2026 16:37:36 +0000 (0:00:00.738) 0:00:02.793 **** 2026-02-25 16:37:37.273066 | instance | ok: [instance] 2026-02-25 16:37:37.273321 | instance | 2026-02-25 16:37:37.273601 | instance | TASK [Install depedencies] ***************************************************** 2026-02-25 16:37:37.273880 | instance | Wednesday 25 February 2026 16:37:37 +0000 (0:00:00.687) 0:00:03.480 **** 2026-02-25 16:37:56.119495 | instance | changed: [instance] 2026-02-25 16:37:56.119625 | instance | 2026-02-25 16:37:56.120001 | instance | TASK [Start up service] ******************************************************** 2026-02-25 16:37:56.120275 | instance | Wednesday 25 February 2026 16:37:56 +0000 (0:00:18.846) 0:00:22.327 **** 2026-02-25 16:37:56.632254 | instance | ok: [instance] 2026-02-25 16:37:56.632506 | instance | 2026-02-25 16:37:56.632780 | instance | TASK [Generate lvm.conf] ******************************************************* 2026-02-25 16:37:56.633046 | instance | Wednesday 25 February 2026 16:37:56 +0000 (0:00:00.513) 0:00:22.840 **** 2026-02-25 16:37:56.906499 | instance | ok: [instance] 2026-02-25 16:37:56.906742 | instance | 2026-02-25 16:37:56.907011 | instance | TASK [Write /etc/lvm/lvm.conf] ************************************************* 2026-02-25 16:37:56.907275 | instance | Wednesday 25 February 2026 16:37:56 +0000 (0:00:00.274) 0:00:23.114 **** 2026-02-25 16:37:57.536608 | instance | changed: [instance] 2026-02-25 16:37:57.536895 | instance | 2026-02-25 16:37:57.537182 | instance | TASK [Get list of all loopback devices] **************************************** 2026-02-25 16:37:57.537466 | instance | Wednesday 25 February 2026 16:37:57 +0000 (0:00:00.630) 0:00:23.744 **** 2026-02-25 16:37:57.720428 | instance | ok: [instance] 2026-02-25 16:37:57.720702 | instance | 2026-02-25 16:37:57.721039 | instance | TASK [Fail if there is any existing loopback devices] ************************** 2026-02-25 16:37:57.721350 | instance | Wednesday 25 February 2026 16:37:57 +0000 (0:00:00.183) 0:00:23.928 **** 2026-02-25 16:37:57.741253 | instance | skipping: [instance] 2026-02-25 16:37:57.741527 | instance | 2026-02-25 16:37:57.741821 | instance | TASK [Create devices for Ceph] ************************************************* 2026-02-25 16:37:57.742119 | instance | Wednesday 25 February 2026 16:37:57 +0000 (0:00:00.020) 0:00:23.949 **** 2026-02-25 16:37:58.273059 | instance | changed: [instance] => (item=osd0) 2026-02-25 16:37:58.273391 | instance | changed: [instance] => (item=osd1) 2026-02-25 16:37:58.273799 | instance | changed: [instance] => (item=osd2) 2026-02-25 16:37:58.274053 | instance | 2026-02-25 16:37:58.274333 | instance | TASK [Set permissions on loopback devices] ************************************* 2026-02-25 16:37:58.274612 | instance | Wednesday 25 February 2026 16:37:58 +0000 (0:00:00.531) 0:00:24.480 **** 2026-02-25 16:37:58.852526 | instance | changed: [instance] => (item=osd0) 2026-02-25 16:37:58.852703 | instance | changed: [instance] => (item=osd1) 2026-02-25 16:37:58.852897 | instance | changed: [instance] => (item=osd2) 2026-02-25 16:37:58.853070 | instance | 2026-02-25 16:37:58.853254 | instance | TASK [Start loop devices] ****************************************************** 2026-02-25 16:37:58.853440 | instance | Wednesday 25 February 2026 16:37:58 +0000 (0:00:00.579) 0:00:25.060 **** 2026-02-25 16:37:59.499701 | instance | changed: [instance] => (item=osd0) 2026-02-25 16:37:59.499987 | instance | changed: [instance] => (item=osd1) 2026-02-25 16:37:59.500286 | instance | changed: [instance] => (item=osd2) 2026-02-25 16:37:59.500572 | instance | 2026-02-25 16:37:59.500878 | instance | TASK [Create a volume group for each loop device] ****************************** 2026-02-25 16:37:59.501254 | instance | Wednesday 25 February 2026 16:37:59 +0000 (0:00:00.646) 0:00:25.707 **** 2026-02-25 16:38:02.445960 | instance | changed: [instance] => (item=osd0) 2026-02-25 16:38:02.446061 | instance | changed: [instance] => (item=osd1) 2026-02-25 16:38:02.446227 | instance | changed: [instance] => (item=osd2) 2026-02-25 16:38:02.446415 | instance | 2026-02-25 16:38:02.446624 | instance | TASK [Create a logical volume for each loop device] **************************** 2026-02-25 16:38:02.446736 | instance | Wednesday 25 February 2026 16:38:02 +0000 (0:00:02.946) 0:00:28.653 **** 2026-02-25 16:38:04.257532 | instance | changed: [instance] => (item=ceph-instance-osd0) 2026-02-25 16:38:04.257725 | instance | changed: [instance] => (item=ceph-instance-osd1) 2026-02-25 16:38:04.258003 | instance | changed: [instance] => (item=ceph-instance-osd2) 2026-02-25 16:38:04.258263 | instance | 2026-02-25 16:38:04.258572 | instance | PLAY RECAP ********************************************************************* 2026-02-25 16:38:04.258873 | instance | instance : ok=15 changed=9 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0 2026-02-25 16:38:04.259132 | instance | 2026-02-25 16:38:04.259413 | instance | Wednesday 25 February 2026 16:38:04 +0000 (0:00:01.811) 0:00:30.465 **** 2026-02-25 16:38:04.259682 | instance | =============================================================================== 2026-02-25 16:38:04.259955 | instance | Install depedencies ---------------------------------------------------- 18.85s 2026-02-25 16:38:04.260233 | instance | Create a volume group for each loop device ------------------------------ 2.95s 2026-02-25 16:38:04.260539 | instance | Create a logical volume for each loop device ---------------------------- 1.81s 2026-02-25 16:38:04.260811 | instance | Gathering Facts --------------------------------------------------------- 1.13s 2026-02-25 16:38:04.261088 | instance | Purge "snapd" package --------------------------------------------------- 0.74s 2026-02-25 16:38:04.261360 | instance | Gathering Facts --------------------------------------------------------- 0.69s 2026-02-25 16:38:04.261633 | instance | Start loop devices ------------------------------------------------------ 0.65s 2026-02-25 16:38:04.261904 | instance | Configure short hostname ------------------------------------------------ 0.64s 2026-02-25 16:38:04.262179 | instance | Write /etc/lvm/lvm.conf ------------------------------------------------- 0.63s 2026-02-25 16:38:04.262484 | instance | Set permissions on loopback devices ------------------------------------- 0.58s 2026-02-25 16:38:04.262760 | instance | Create devices for Ceph ------------------------------------------------- 0.53s 2026-02-25 16:38:04.263034 | instance | Start up service -------------------------------------------------------- 0.51s 2026-02-25 16:38:04.263306 | instance | Generate lvm.conf ------------------------------------------------------- 0.27s 2026-02-25 16:38:04.263581 | instance | Ensure hostname inside hosts file --------------------------------------- 0.26s 2026-02-25 16:38:04.263854 | instance | Get list of all loopback devices ---------------------------------------- 0.18s 2026-02-25 16:38:04.264131 | instance | Fail if there is any existing loopback devices -------------------------- 0.02s 2026-02-25 16:38:04.327500 | instance | INFO [csi > prepare] Executed: Successful 2026-02-25 16:38:04.338987 | instance | INFO [csi > converge] Executing 2026-02-25 16:38:05.672168 | instance | 2026-02-25 16:38:05.672252 | instance | PLAY [Debug CSI driver value] ************************************************** 2026-02-25 16:38:05.672411 | instance | 2026-02-25 16:38:05.672556 | instance | TASK [Gathering Facts] ********************************************************* 2026-02-25 16:38:05.672706 | instance | Wednesday 25 February 2026 16:38:05 +0000 (0:00:00.007) 0:00:00.007 **** 2026-02-25 16:38:06.594781 | instance | [WARNING]: Platform linux on host instance is using the discovered Python 2026-02-25 16:38:06.595077 | instance | interpreter at /usr/bin/python3.10, but future installation of another Python 2026-02-25 16:38:06.595382 | instance | interpreter could change the meaning of that path. See 2026-02-25 16:38:06.595678 | instance | https://docs.ansible.com/ansible- 2026-02-25 16:38:06.595971 | instance | core/2.17/reference_appendices/interpreter_discovery.html for more information. 2026-02-25 16:38:06.605909 | instance | ok: [instance] 2026-02-25 16:38:06.606157 | instance | 2026-02-25 16:38:06.606454 | instance | TASK [Display CSI driver value and environment variable] *********************** 2026-02-25 16:38:06.606731 | instance | Wednesday 25 February 2026 16:38:06 +0000 (0:00:00.933) 0:00:00.941 **** 2026-02-25 16:38:06.648991 | instance | ok: [instance] => { 2026-02-25 16:38:06.649234 | instance | "msg": "csi_driver=rbd, MOLECULE_CSI_DRIVER=" 2026-02-25 16:38:06.649544 | instance | } 2026-02-25 16:38:06.649835 | instance | 2026-02-25 16:38:06.650123 | instance | PLAY [all] ********************************************************************* 2026-02-25 16:38:06.650363 | instance | 2026-02-25 16:38:06.650664 | instance | TASK [Gathering Facts] ********************************************************* 2026-02-25 16:38:06.650928 | instance | Wednesday 25 February 2026 16:38:06 +0000 (0:00:00.043) 0:00:00.984 **** 2026-02-25 16:38:07.476393 | instance | ok: [instance] 2026-02-25 16:38:07.476607 | instance | 2026-02-25 16:38:07.476892 | instance | TASK [Set a fact with the "atmosphere_images" for other plays] ***************** 2026-02-25 16:38:07.477175 | instance | Wednesday 25 February 2026 16:38:07 +0000 (0:00:00.827) 0:00:01.811 **** 2026-02-25 16:38:07.656609 | instance | ok: [instance] 2026-02-25 16:38:07.656836 | instance | 2026-02-25 16:38:07.657129 | instance | PLAY [Deploy Ceph monitors & managers] ***************************************** 2026-02-25 16:38:07.657401 | instance | 2026-02-25 16:38:07.657761 | instance | TASK [Gathering Facts] ********************************************************* 2026-02-25 16:38:07.658171 | instance | Wednesday 25 February 2026 16:38:07 +0000 (0:00:00.180) 0:00:01.991 **** 2026-02-25 16:38:08.498532 | instance | ok: [instance] 2026-02-25 16:38:08.498956 | instance | 2026-02-25 16:38:08.499327 | instance | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-02-25 16:38:08.499679 | instance | Wednesday 25 February 2026 16:38:08 +0000 (0:00:00.841) 0:00:02.833 **** 2026-02-25 16:38:08.771163 | instance | ok: [instance] 2026-02-25 16:38:08.771540 | instance | 2026-02-25 16:38:08.771882 | instance | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-02-25 16:38:08.772216 | instance | Wednesday 25 February 2026 16:38:08 +0000 (0:00:00.271) 0:00:03.105 **** 2026-02-25 16:38:08.808627 | instance | skipping: [instance] 2026-02-25 16:38:08.808810 | instance | 2026-02-25 16:38:08.808966 | instance | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-02-25 16:38:08.809117 | instance | Wednesday 25 February 2026 16:38:08 +0000 (0:00:00.038) 0:00:03.143 **** 2026-02-25 16:38:09.100319 | instance | changed: [instance] 2026-02-25 16:38:09.100352 | instance | 2026-02-25 16:38:09.100358 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-02-25 16:38:09.100363 | instance | Wednesday 25 February 2026 16:38:09 +0000 (0:00:00.292) 0:00:03.435 **** 2026-02-25 16:38:09.162459 | instance | ok: [instance] => { 2026-02-25 16:38:09.162510 | instance | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.0/runc.amd64" 2026-02-25 16:38:09.163269 | instance | } 2026-02-25 16:38:09.163290 | instance | 2026-02-25 16:38:09.163297 | instance | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-02-25 16:38:09.163303 | instance | Wednesday 25 February 2026 16:38:09 +0000 (0:00:00.061) 0:00:03.497 **** 2026-02-25 16:38:09.817730 | instance | changed: [instance] 2026-02-25 16:38:09.817813 | instance | 2026-02-25 16:38:09.818063 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-02-25 16:38:09.818094 | instance | Wednesday 25 February 2026 16:38:09 +0000 (0:00:00.655) 0:00:04.153 **** 2026-02-25 16:38:09.869947 | instance | skipping: [instance] 2026-02-25 16:38:09.870006 | instance | 2026-02-25 16:38:09.870264 | instance | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-02-25 16:38:09.870304 | instance | Wednesday 25 February 2026 16:38:09 +0000 (0:00:00.052) 0:00:04.205 **** 2026-02-25 16:38:09.920514 | instance | skipping: [instance] 2026-02-25 16:38:09.920598 | instance | 2026-02-25 16:38:09.920854 | instance | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-02-25 16:38:09.920881 | instance | Wednesday 25 February 2026 16:38:09 +0000 (0:00:00.050) 0:00:04.256 **** 2026-02-25 16:38:10.119687 | instance | ok: [instance] 2026-02-25 16:38:10.119784 | instance | 2026-02-25 16:38:10.120072 | instance | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-02-25 16:38:10.120111 | instance | Wednesday 25 February 2026 16:38:10 +0000 (0:00:00.199) 0:00:04.455 **** 2026-02-25 16:38:11.257835 | instance | ok: [instance] 2026-02-25 16:38:11.257900 | instance | 2026-02-25 16:38:11.258046 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-02-25 16:38:11.258237 | instance | Wednesday 25 February 2026 16:38:11 +0000 (0:00:01.138) 0:00:05.593 **** 2026-02-25 16:38:11.322944 | instance | ok: [instance] => { 2026-02-25 16:38:11.323059 | instance | "msg": "https://github.com/containerd/containerd/releases/download/v2.2.0/containerd-2.2.0-linux-amd64.tar.gz" 2026-02-25 16:38:11.323256 | instance | } 2026-02-25 16:38:11.323406 | instance | 2026-02-25 16:38:11.323567 | instance | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-02-25 16:38:11.323721 | instance | Wednesday 25 February 2026 16:38:11 +0000 (0:00:00.064) 0:00:05.658 **** 2026-02-25 16:38:12.055026 | instance | changed: [instance] 2026-02-25 16:38:12.055091 | instance | 2026-02-25 16:38:12.055296 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-02-25 16:38:12.055660 | instance | Wednesday 25 February 2026 16:38:12 +0000 (0:00:00.731) 0:00:06.389 **** 2026-02-25 16:38:14.896568 | instance | changed: [instance] 2026-02-25 16:38:14.896670 | instance | 2026-02-25 16:38:14.896716 | instance | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-02-25 16:38:14.896890 | instance | Wednesday 25 February 2026 16:38:14 +0000 (0:00:02.842) 0:00:09.232 **** 2026-02-25 16:38:14.928655 | instance | skipping: [instance] 2026-02-25 16:38:14.929052 | instance | 2026-02-25 16:38:14.929092 | instance | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-02-25 16:38:14.929098 | instance | Wednesday 25 February 2026 16:38:14 +0000 (0:00:00.031) 0:00:09.264 **** 2026-02-25 16:38:14.963959 | instance | skipping: [instance] 2026-02-25 16:38:14.964514 | instance | 2026-02-25 16:38:14.964541 | instance | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-02-25 16:38:14.964547 | instance | Wednesday 25 February 2026 16:38:14 +0000 (0:00:00.035) 0:00:09.299 **** 2026-02-25 16:38:14.993893 | instance | skipping: [instance] 2026-02-25 16:38:14.994296 | instance | 2026-02-25 16:38:14.994482 | instance | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-02-25 16:38:14.994490 | instance | Wednesday 25 February 2026 16:38:14 +0000 (0:00:00.029) 0:00:09.329 **** 2026-02-25 16:38:20.098741 | instance | changed: [instance] 2026-02-25 16:38:20.098829 | instance | 2026-02-25 16:38:20.099282 | instance | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-02-25 16:38:20.099323 | instance | Wednesday 25 February 2026 16:38:20 +0000 (0:00:05.104) 0:00:14.433 **** 2026-02-25 16:38:20.654474 | instance | changed: [instance] 2026-02-25 16:38:20.654994 | instance | 2026-02-25 16:38:20.655017 | instance | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-02-25 16:38:20.655025 | instance | Wednesday 25 February 2026 16:38:20 +0000 (0:00:00.555) 0:00:14.989 **** 2026-02-25 16:38:21.543684 | instance | changed: [instance] => (item={'path': '/etc/containerd'}) 2026-02-25 16:38:21.544516 | instance | changed: [instance] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-02-25 16:38:21.544540 | instance | changed: [instance] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-02-25 16:38:21.544545 | instance | changed: [instance] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-02-25 16:38:21.544550 | instance | changed: [instance] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-02-25 16:38:21.544555 | instance | 2026-02-25 16:38:21.544559 | instance | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-02-25 16:38:21.544563 | instance | Wednesday 25 February 2026 16:38:21 +0000 (0:00:00.889) 0:00:15.879 **** 2026-02-25 16:38:22.079958 | instance | changed: [instance] 2026-02-25 16:38:22.080705 | instance | 2026-02-25 16:38:22.080844 | instance | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-02-25 16:38:22.080851 | instance | Wednesday 25 February 2026 16:38:22 +0000 (0:00:00.519) 0:00:16.399 **** 2026-02-25 16:38:22.080855 | instance | 2026-02-25 16:38:22.080859 | instance | RUNNING HANDLER [vexxhost.containers.containerd : Reload systemd] ************** 2026-02-25 16:38:22.080863 | instance | Wednesday 25 February 2026 16:38:22 +0000 (0:00:00.016) 0:00:16.415 **** 2026-02-25 16:38:22.945549 | instance | ok: [instance] 2026-02-25 16:38:22.945682 | instance | 2026-02-25 16:38:22.946062 | instance | RUNNING HANDLER [vexxhost.containers.containerd : Restart containerd] ********** 2026-02-25 16:38:22.946082 | instance | Wednesday 25 February 2026 16:38:22 +0000 (0:00:00.865) 0:00:17.281 **** 2026-02-25 16:38:23.374270 | instance | changed: [instance] 2026-02-25 16:38:23.374362 | instance | 2026-02-25 16:38:23.374528 | instance | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-02-25 16:38:23.374661 | instance | Wednesday 25 February 2026 16:38:23 +0000 (0:00:00.428) 0:00:17.709 **** 2026-02-25 16:38:23.900417 | instance | changed: [instance] 2026-02-25 16:38:23.900501 | instance | 2026-02-25 16:38:23.900617 | instance | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-02-25 16:38:23.900738 | instance | Wednesday 25 February 2026 16:38:23 +0000 (0:00:00.526) 0:00:18.236 **** 2026-02-25 16:38:24.095502 | instance | ok: [instance] 2026-02-25 16:38:24.095728 | instance | 2026-02-25 16:38:24.096019 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-02-25 16:38:24.096315 | instance | Wednesday 25 February 2026 16:38:24 +0000 (0:00:00.194) 0:00:18.430 **** 2026-02-25 16:38:24.141576 | instance | ok: [instance] => { 2026-02-25 16:38:24.141715 | instance | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-02-25 16:38:24.141885 | instance | } 2026-02-25 16:38:24.142047 | instance | 2026-02-25 16:38:24.142219 | instance | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-02-25 16:38:24.142388 | instance | Wednesday 25 February 2026 16:38:24 +0000 (0:00:00.046) 0:00:18.477 **** 2026-02-25 16:38:25.148277 | instance | changed: [instance] 2026-02-25 16:38:25.148341 | instance | 2026-02-25 16:38:25.148354 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-02-25 16:38:25.148365 | instance | Wednesday 25 February 2026 16:38:25 +0000 (0:00:01.005) 0:00:19.482 **** 2026-02-25 16:38:29.394667 | instance | changed: [instance] 2026-02-25 16:38:29.394755 | instance | 2026-02-25 16:38:29.395033 | instance | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-02-25 16:38:29.395083 | instance | Wednesday 25 February 2026 16:38:29 +0000 (0:00:04.247) 0:00:23.730 **** 2026-02-25 16:38:30.377996 | instance | ok: [instance] 2026-02-25 16:38:30.378072 | instance | 2026-02-25 16:38:30.378313 | instance | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-02-25 16:38:30.378348 | instance | Wednesday 25 February 2026 16:38:30 +0000 (0:00:00.983) 0:00:24.713 **** 2026-02-25 16:38:30.711855 | instance | changed: [instance] 2026-02-25 16:38:30.711917 | instance | 2026-02-25 16:38:30.712208 | instance | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-02-25 16:38:30.712258 | instance | Wednesday 25 February 2026 16:38:30 +0000 (0:00:00.333) 0:00:25.047 **** 2026-02-25 16:38:31.166451 | instance | changed: [instance] 2026-02-25 16:38:31.166739 | instance | 2026-02-25 16:38:31.166752 | instance | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-02-25 16:38:31.166840 | instance | Wednesday 25 February 2026 16:38:31 +0000 (0:00:00.454) 0:00:25.502 **** 2026-02-25 16:38:31.725474 | instance | changed: [instance] => (item={'path': '/etc/docker'}) 2026-02-25 16:38:31.725519 | instance | changed: [instance] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-02-25 16:38:31.725525 | instance | changed: [instance] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-02-25 16:38:31.725529 | instance | 2026-02-25 16:38:31.725539 | instance | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-02-25 16:38:31.725543 | instance | Wednesday 25 February 2026 16:38:31 +0000 (0:00:00.558) 0:00:26.060 **** 2026-02-25 16:38:32.189946 | instance | changed: [instance] 2026-02-25 16:38:32.190301 | instance | 2026-02-25 16:38:32.190319 | instance | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-02-25 16:38:32.190324 | instance | Wednesday 25 February 2026 16:38:32 +0000 (0:00:00.465) 0:00:26.525 **** 2026-02-25 16:38:32.665319 | instance | changed: [instance] 2026-02-25 16:38:32.665395 | instance | 2026-02-25 16:38:32.666070 | instance | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-02-25 16:38:32.666118 | instance | Wednesday 25 February 2026 16:38:32 +0000 (0:00:00.456) 0:00:26.982 **** 2026-02-25 16:38:32.666124 | instance | 2026-02-25 16:38:32.666129 | instance | RUNNING HANDLER [vexxhost.containers.containerd : Reload systemd] ************** 2026-02-25 16:38:32.666133 | instance | Wednesday 25 February 2026 16:38:32 +0000 (0:00:00.018) 0:00:27.000 **** 2026-02-25 16:38:33.341210 | instance | ok: [instance] 2026-02-25 16:38:33.341269 | instance | 2026-02-25 16:38:33.341564 | instance | RUNNING HANDLER [vexxhost.containers.docker : Restart docker] ****************** 2026-02-25 16:38:33.341606 | instance | Wednesday 25 February 2026 16:38:33 +0000 (0:00:00.675) 0:00:27.676 **** 2026-02-25 16:38:34.073606 | instance | changed: [instance] 2026-02-25 16:38:34.073709 | instance | 2026-02-25 16:38:34.073988 | instance | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-02-25 16:38:34.074027 | instance | Wednesday 25 February 2026 16:38:34 +0000 (0:00:00.732) 0:00:28.409 **** 2026-02-25 16:38:34.628827 | instance | changed: [instance] 2026-02-25 16:38:34.629181 | instance | 2026-02-25 16:38:34.629197 | instance | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-02-25 16:38:34.629202 | instance | Wednesday 25 February 2026 16:38:34 +0000 (0:00:00.555) 0:00:28.964 **** 2026-02-25 16:38:34.677037 | instance | ok: [instance] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/ubuntu-22.04.yml) 2026-02-25 16:38:34.677125 | instance | 2026-02-25 16:38:34.677406 | instance | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-02-25 16:38:34.677444 | instance | Wednesday 25 February 2026 16:38:34 +0000 (0:00:00.048) 0:00:29.012 **** 2026-02-25 16:38:39.247953 | instance | changed: [instance] 2026-02-25 16:38:39.248018 | instance | 2026-02-25 16:38:39.248291 | instance | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-02-25 16:38:39.248307 | instance | Wednesday 25 February 2026 16:38:39 +0000 (0:00:04.570) 0:00:33.583 **** 2026-02-25 16:38:39.885729 | instance | ok: [instance] => (item=chronyd) 2026-02-25 16:38:39.885914 | instance | ok: [instance] => (item=sshd) 2026-02-25 16:38:39.885927 | instance | 2026-02-25 16:38:39.886259 | instance | TASK [vexxhost.ceph.cephadm : Download "cephadm"] ****************************** 2026-02-25 16:38:39.886329 | instance | Wednesday 25 February 2026 16:38:39 +0000 (0:00:00.637) 0:00:34.221 **** 2026-02-25 16:38:40.165226 | instance | changed: [instance] 2026-02-25 16:38:40.165299 | instance | 2026-02-25 16:38:40.165586 | instance | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-02-25 16:38:40.165626 | instance | Wednesday 25 February 2026 16:38:40 +0000 (0:00:00.279) 0:00:34.500 **** 2026-02-25 16:38:40.349329 | instance | ok: [instance] 2026-02-25 16:38:40.349732 | instance | 2026-02-25 16:38:40.349748 | instance | TASK [vexxhost.ceph.cephadm : Ensure "cephadm" user is present] **************** 2026-02-25 16:38:40.349765 | instance | Wednesday 25 February 2026 16:38:40 +0000 (0:00:00.183) 0:00:34.684 **** 2026-02-25 16:38:40.780337 | instance | changed: [instance] 2026-02-25 16:38:40.780873 | instance | 2026-02-25 16:38:40.780918 | instance | TASK [vexxhost.ceph.cephadm : Allow "cephadm" user to have passwordless sudo] *** 2026-02-25 16:38:40.780925 | instance | Wednesday 25 February 2026 16:38:40 +0000 (0:00:00.431) 0:00:35.115 **** 2026-02-25 16:38:41.093079 | instance | changed: [instance] 2026-02-25 16:38:41.093189 | instance | 2026-02-25 16:38:41.093489 | instance | TASK [vexxhost.ceph.mon : Get `cephadm ls` status] ***************************** 2026-02-25 16:38:41.093527 | instance | Wednesday 25 February 2026 16:38:41 +0000 (0:00:00.313) 0:00:35.428 **** 2026-02-25 16:38:42.703477 | instance | ok: [instance] 2026-02-25 16:38:42.703571 | instance | 2026-02-25 16:38:42.703826 | instance | TASK [vexxhost.ceph.mon : Parse the `cephadm ls` output] *********************** 2026-02-25 16:38:42.703841 | instance | Wednesday 25 February 2026 16:38:42 +0000 (0:00:01.610) 0:00:37.039 **** 2026-02-25 16:38:42.753905 | instance | ok: [instance] 2026-02-25 16:38:42.754009 | instance | 2026-02-25 16:38:42.754261 | instance | TASK [vexxhost.ceph.mon : Assimilate existing configs in `ceph.conf`] ********** 2026-02-25 16:38:42.754277 | instance | Wednesday 25 February 2026 16:38:42 +0000 (0:00:00.050) 0:00:37.089 **** 2026-02-25 16:38:42.789240 | instance | skipping: [instance] 2026-02-25 16:38:42.789353 | instance | 2026-02-25 16:38:42.789467 | instance | TASK [vexxhost.ceph.mon : Adopt monitor to cluster] **************************** 2026-02-25 16:38:42.789595 | instance | Wednesday 25 February 2026 16:38:42 +0000 (0:00:00.035) 0:00:37.125 **** 2026-02-25 16:38:42.825863 | instance | skipping: [instance] 2026-02-25 16:38:42.825953 | instance | 2026-02-25 16:38:42.826236 | instance | TASK [vexxhost.ceph.mon : Adopt manager to cluster] **************************** 2026-02-25 16:38:42.826251 | instance | Wednesday 25 February 2026 16:38:42 +0000 (0:00:00.036) 0:00:37.161 **** 2026-02-25 16:38:42.860090 | instance | skipping: [instance] 2026-02-25 16:38:42.860151 | instance | 2026-02-25 16:38:42.860415 | instance | TASK [vexxhost.ceph.mon : Enable "cephadm" mgr module] ************************* 2026-02-25 16:38:42.860430 | instance | Wednesday 25 February 2026 16:38:42 +0000 (0:00:00.034) 0:00:37.195 **** 2026-02-25 16:38:42.894693 | instance | skipping: [instance] 2026-02-25 16:38:42.894771 | instance | 2026-02-25 16:38:42.895043 | instance | TASK [vexxhost.ceph.mon : Set orchestrator backend to "cephadm"] *************** 2026-02-25 16:38:42.895082 | instance | Wednesday 25 February 2026 16:38:42 +0000 (0:00:00.034) 0:00:37.230 **** 2026-02-25 16:38:42.929132 | instance | skipping: [instance] 2026-02-25 16:38:42.929214 | instance | 2026-02-25 16:38:42.929341 | instance | TASK [vexxhost.ceph.mon : Use `cephadm` user for cephadm] ********************** 2026-02-25 16:38:42.929529 | instance | Wednesday 25 February 2026 16:38:42 +0000 (0:00:00.034) 0:00:37.264 **** 2026-02-25 16:38:42.962650 | instance | skipping: [instance] 2026-02-25 16:38:42.962772 | instance | 2026-02-25 16:38:42.962997 | instance | TASK [vexxhost.ceph.mon : Generate "cephadm" key] ****************************** 2026-02-25 16:38:42.963153 | instance | Wednesday 25 February 2026 16:38:42 +0000 (0:00:00.033) 0:00:37.298 **** 2026-02-25 16:38:43.000093 | instance | skipping: [instance] 2026-02-25 16:38:43.000209 | instance | 2026-02-25 16:38:43.000401 | instance | TASK [vexxhost.ceph.mon : Set Ceph Monitor IP address] ************************* 2026-02-25 16:38:43.000557 | instance | Wednesday 25 February 2026 16:38:42 +0000 (0:00:00.037) 0:00:37.335 **** 2026-02-25 16:38:43.108354 | instance | ok: [instance] 2026-02-25 16:38:43.108429 | instance | 2026-02-25 16:38:43.108521 | instance | TASK [vexxhost.ceph.mon : Check if any node is bootstrapped] ******************* 2026-02-25 16:38:43.108695 | instance | Wednesday 25 February 2026 16:38:43 +0000 (0:00:00.108) 0:00:37.443 **** 2026-02-25 16:38:43.322720 | instance | ok: [instance] => (item=instance) 2026-02-25 16:38:43.322941 | instance | 2026-02-25 16:38:43.323103 | instance | TASK [vexxhost.ceph.mon : Select pre-existing bootstrap node if exists] ******** 2026-02-25 16:38:43.323187 | instance | Wednesday 25 February 2026 16:38:43 +0000 (0:00:00.214) 0:00:37.658 **** 2026-02-25 16:38:43.369887 | instance | ok: [instance] 2026-02-25 16:38:43.370269 | instance | 2026-02-25 16:38:43.370442 | instance | TASK [vexxhost.ceph.mon : Bootstrap cluster] *********************************** 2026-02-25 16:38:43.370459 | instance | Wednesday 25 February 2026 16:38:43 +0000 (0:00:00.046) 0:00:37.705 **** 2026-02-25 16:38:43.438482 | instance | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/mon/tasks/bootstrap-ceph.yml for instance 2026-02-25 16:38:43.438858 | instance | 2026-02-25 16:38:43.438887 | instance | TASK [vexxhost.ceph.mon : Generate temporary file for "ceph.conf"] ************* 2026-02-25 16:38:43.438893 | instance | Wednesday 25 February 2026 16:38:43 +0000 (0:00:00.068) 0:00:37.773 **** 2026-02-25 16:38:43.737283 | instance | changed: [instance] 2026-02-25 16:38:43.737372 | instance | 2026-02-25 16:38:43.737636 | instance | TASK [vexxhost.ceph.mon : Include extra configuration values] ****************** 2026-02-25 16:38:43.737651 | instance | Wednesday 25 February 2026 16:38:43 +0000 (0:00:00.298) 0:00:38.072 **** 2026-02-25 16:38:44.385198 | instance | changed: [instance] => (item={'option': 'mon allow pool size one', 'section': 'global', 'value': True}) 2026-02-25 16:38:44.385823 | instance | changed: [instance] => (item={'option': 'osd crush chooseleaf type', 'section': 'global', 'value': 0}) 2026-02-25 16:38:44.385859 | instance | changed: [instance] => (item={'option': 'auth allow insecure global id reclaim', 'section': 'mon', 'value': False}) 2026-02-25 16:38:44.385865 | instance | 2026-02-25 16:38:44.385870 | instance | TASK [vexxhost.ceph.mon : Run Bootstrap coomand] ******************************* 2026-02-25 16:38:44.385874 | instance | Wednesday 25 February 2026 16:38:44 +0000 (0:00:00.647) 0:00:38.720 **** 2026-02-25 16:38:46.177120 | instance | fatal: [instance]: FAILED! => {"changed": false, "cmd": ["cephadm", "bootstrap", "--fsid", "4837cbf8-4f90-4300-b3f6-726c9b9f89b4", "--mon-ip", "199.204.45.162", "--cluster-network", "0.0.0.0/0", "--ssh-user", "cephadm", "--config", "/tmp/ceph_yscx59tm.conf", "--skip-monitoring-stack"], "delta": "0:00:01.611607", "end": "2026-02-25 16:38:46.151658", "msg": "non-zero return code", "rc": 1, "start": "2026-02-25 16:38:44.540051", "stderr": "Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts.\nThe cluster CIDR network 0.0.0.0/0 is not configured locally.\nError: Failed command: /usr/bin/docker pull quay.io/ceph/ceph:v18.2.1\nERROR: Failed command: /usr/bin/docker pull quay.io/ceph/ceph:v18.2.1", "stderr_lines": ["Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts.", "The cluster CIDR network 0.0.0.0/0 is not configured locally.", "Error: Failed command: /usr/bin/docker pull quay.io/ceph/ceph:v18.2.1", "ERROR: Failed command: /usr/bin/docker pull quay.io/ceph/ceph:v18.2.1"], "stdout": "Creating directory /etc/ceph for ceph.conf\nVerifying ssh connectivity using standard pubkey authentication ...\nAdding key to cephadm@localhost authorized_keys...\nVerifying podman|docker is present...\nVerifying lvm2 is present...\nVerifying time synchronization is in place...\nUnit chrony.service is enabled and running\nRepeating the final host check...\ndocker (/usr/bin/docker) is present\nsystemctl is present\nlvcreate is present\nUnit chrony.service is enabled and running\nHost looks OK\nCluster fsid: 4837cbf8-4f90-4300-b3f6-726c9b9f89b4\nVerifying IP 199.204.45.162 port 3300 ...\nVerifying IP 199.204.45.162 port 6789 ...\nMon IP `199.204.45.162` is in CIDR network `162.253.52.0/24`\nMon IP `199.204.45.162` is in CIDR network `162.253.52.0/24`\nMon IP `199.204.45.162` is in CIDR network `162.253.53.0/24`\nMon IP `199.204.45.162` is in CIDR network `162.253.53.0/24`\nMon IP `199.204.45.162` is in CIDR network `162.253.54.0/24`\nMon IP `199.204.45.162` is in CIDR network `162.253.54.0/24`\nMon IP `199.204.45.162` is in CIDR network `162.253.55.0/24`\nMon IP `199.204.45.162` is in CIDR network `162.253.55.0/24`\nMon IP `199.204.45.162` is in CIDR network `199.19.212.0/24`\nMon IP `199.204.45.162` is in CIDR network `199.19.212.0/24`\nMon IP `199.204.45.162` is in CIDR network `199.19.213.0/24`\nMon IP `199.204.45.162` is in CIDR network `199.19.213.0/24`\nMon IP `199.204.45.162` is in CIDR network `199.19.214.0/24`\nMon IP `199.204.45.162` is in CIDR network `199.19.214.0/24`\nMon IP `199.204.45.162` is in CIDR network `199.19.215.0/24`\nMon IP `199.204.45.162` is in CIDR network `199.19.215.0/24`\nMon IP `199.204.45.162` is in CIDR network `199.204.45.0/24`\nMon IP `199.204.45.162` is in CIDR network `199.204.45.0/24`\nMon IP `199.204.45.162` is in CIDR network `199.204.45.1/32`\nMon IP `199.204.45.162` is in CIDR network `199.204.45.1/32`\nMon IP `199.204.45.162` is in CIDR network `199.204.45.13/32`\nMon IP `199.204.45.162` is in CIDR network `199.204.45.13/32`\nMon IP `199.204.45.162` is in CIDR network `199.204.46.0/24`\nMon IP `199.204.45.162` is in CIDR network `199.204.46.0/24`\nPulling container image quay.io/ceph/ceph:v18.2.1...\nNon-zero exit code 1 from /usr/bin/docker pull quay.io/ceph/ceph:v18.2.1\n/usr/bin/docker: stdout v18.2.1: Pulling from ceph/ceph\n/usr/bin/docker: stderr error unmarshalling content: unexpected end of JSON input\n\n\n\t***************\n\tCephadm hit an issue during cluster installation. Current cluster files will NOT BE DELETED automatically to change\n\tthis behaviour you can pass the --cleanup-on-failure. To remove this broken cluster manually please run:\n\n\t > cephadm rm-cluster --force --fsid 4837cbf8-4f90-4300-b3f6-726c9b9f89b4\n\n\tin case of any previous broken installation user must use the rm-cluster command to delete the broken cluster:\n\n\t > cephadm rm-cluster --force --zap-osds --fsid \n\n\tfor more information please refer to https://docs.ceph.com/en/latest/cephadm/operations/#purging-a-cluster\n\t***************", "stdout_lines": ["Creating directory /etc/ceph for ceph.conf", "Verifying ssh connectivity using standard pubkey authentication ...", "Adding key to cephadm@localhost authorized_keys...", "Verifying podman|docker is present...", "Verifying lvm2 is present...", "Verifying time synchronization is in place...", "Unit chrony.service is enabled and running", "Repeating the final host check...", "docker (/usr/bin/docker) is present", "systemctl is present", "lvcreate is present", "Unit chrony.service is enabled and running", "Host looks OK", "Cluster fsid: 4837cbf8-4f90-4300-b3f6-726c9b9f89b4", "Verifying IP 199.204.45.162 port 3300 ...", "Verifying IP 199.204.45.162 port 6789 ...", "Mon IP `199.204.45.162` is in CIDR network `162.253.52.0/24`", "Mon IP `199.204.45.162` is in CIDR network `162.253.52.0/24`", "Mon IP `199.204.45.162` is in CIDR network `162.253.53.0/24`", "Mon IP `199.204.45.162` is in CIDR network `162.253.53.0/24`", "Mon IP `199.204.45.162` is in CIDR network `162.253.54.0/24`", "Mon IP `199.204.45.162` is in CIDR network `162.253.54.0/24`", "Mon IP `199.204.45.162` is in CIDR network `162.253.55.0/24`", "Mon IP `199.204.45.162` is in CIDR network `162.253.55.0/24`", "Mon IP `199.204.45.162` is in CIDR network `199.19.212.0/24`", "Mon IP `199.204.45.162` is in CIDR network `199.19.212.0/24`", "Mon IP `199.204.45.162` is in CIDR network `199.19.213.0/24`", "Mon IP `199.204.45.162` is in CIDR network `199.19.213.0/24`", "Mon IP `199.204.45.162` is in CIDR network `199.19.214.0/24`", "Mon IP `199.204.45.162` is in CIDR network `199.19.214.0/24`", "Mon IP `199.204.45.162` is in CIDR network `199.19.215.0/24`", "Mon IP `199.204.45.162` is in CIDR network `199.19.215.0/24`", "Mon IP `199.204.45.162` is in CIDR network `199.204.45.0/24`", "Mon IP `199.204.45.162` is in CIDR network `199.204.45.0/24`", "Mon IP `199.204.45.162` is in CIDR network `199.204.45.1/32`", "Mon IP `199.204.45.162` is in CIDR network `199.204.45.1/32`", "Mon IP `199.204.45.162` is in CIDR network `199.204.45.13/32`", "Mon IP `199.204.45.162` is in CIDR network `199.204.45.13/32`", "Mon IP `199.204.45.162` is in CIDR network `199.204.46.0/24`", "Mon IP `199.204.45.162` is in CIDR network `199.204.46.0/24`", "Pulling container image quay.io/ceph/ceph:v18.2.1...", "Non-zero exit code 1 from /usr/bin/docker pull quay.io/ceph/ceph:v18.2.1", "/usr/bin/docker: stdout v18.2.1: Pulling from ceph/ceph", "/usr/bin/docker: stderr error unmarshalling content: unexpected end of JSON input", "", "", "\t***************", "\tCephadm hit an issue during cluster installation. Current cluster files will NOT BE DELETED automatically to change", "\tthis behaviour you can pass the --cleanup-on-failure. To remove this broken cluster manually please run:", "", "\t > cephadm rm-cluster --force --fsid 4837cbf8-4f90-4300-b3f6-726c9b9f89b4", "", "\tin case of any previous broken installation user must use the rm-cluster command to delete the broken cluster:", "", "\t > cephadm rm-cluster --force --zap-osds --fsid ", "", "\tfor more information please refer to https://docs.ceph.com/en/latest/cephadm/operations/#purging-a-cluster", "\t***************"]} 2026-02-25 16:38:46.193081 | instance | 2026-02-25 16:38:46.193195 | instance | TASK [vexxhost.ceph.mon : Remove temporary file for "ceph.conf"] *************** 2026-02-25 16:38:46.193323 | instance | Wednesday 25 February 2026 16:38:46 +0000 (0:00:01.808) 0:00:40.528 **** 2026-02-25 16:38:46.390499 | instance | changed: [instance] 2026-02-25 16:38:46.390557 | instance | 2026-02-25 16:38:46.390574 | instance | PLAY RECAP ********************************************************************* 2026-02-25 16:38:46.390588 | instance | instance : ok=50 changed=26 unreachable=0 failed=1 skipped=13 rescued=0 ignored=0 2026-02-25 16:38:46.394245 | instance | 2026-02-25 16:38:46.394286 | instance | Wednesday 25 February 2026 16:38:46 +0000 (0:00:00.196) 0:00:40.725 **** 2026-02-25 16:38:46.394292 | instance | =============================================================================== 2026-02-25 16:38:46.394297 | instance | vexxhost.containers.containerd : Install AppArmor packages -------------- 5.10s 2026-02-25 16:38:46.394301 | instance | vexxhost.ceph.cephadm : Install packages -------------------------------- 4.57s 2026-02-25 16:38:46.394305 | instance | vexxhost.containers.download_artifact : Extract archive ----------------- 4.25s 2026-02-25 16:38:46.394309 | instance | vexxhost.containers.download_artifact : Extract archive ----------------- 2.84s 2026-02-25 16:38:46.394313 | instance | vexxhost.ceph.mon : Run Bootstrap coomand ------------------------------- 1.81s 2026-02-25 16:38:46.394317 | instance | vexxhost.ceph.mon : Get `cephadm ls` status ----------------------------- 1.61s 2026-02-25 16:38:46.394321 | instance | vexxhost.containers.containerd : Reload systemd ------------------------- 1.54s 2026-02-25 16:38:46.394325 | instance | vexxhost.containers.package : Update state for tar ---------------------- 1.14s 2026-02-25 16:38:46.394329 | instance | vexxhost.containers.download_artifact : Download item ------------------- 1.01s 2026-02-25 16:38:46.394332 | instance | vexxhost.containers.docker : Install AppArmor packages ------------------ 0.98s 2026-02-25 16:38:46.394336 | instance | Gathering Facts --------------------------------------------------------- 0.93s 2026-02-25 16:38:46.394340 | instance | vexxhost.containers.containerd : Create folders for configuration ------- 0.89s 2026-02-25 16:38:46.394344 | instance | Gathering Facts --------------------------------------------------------- 0.84s 2026-02-25 16:38:46.394348 | instance | Gathering Facts --------------------------------------------------------- 0.83s 2026-02-25 16:38:46.394352 | instance | vexxhost.containers.docker : Restart docker ----------------------------- 0.73s 2026-02-25 16:38:46.394355 | instance | vexxhost.containers.download_artifact : Download item ------------------- 0.73s 2026-02-25 16:38:46.394359 | instance | vexxhost.containers.download_artifact : Download item ------------------- 0.66s 2026-02-25 16:38:46.394363 | instance | vexxhost.ceph.mon : Include extra configuration values ------------------ 0.65s 2026-02-25 16:38:46.394367 | instance | vexxhost.ceph.cephadm : Ensure services are started --------------------- 0.64s 2026-02-25 16:38:46.394371 | instance | vexxhost.containers.docker : Create folders for configuration ----------- 0.56s 2026-02-25 16:38:46.472400 | instance | CRITICAL Ansible return code was 2, command was: ansible-playbook --inventory /home/zuul/.ansible/tmp/molecule.v9Wo.csi/inventory --skip-tags molecule-notest,notest --inventory=/home/zuul/src/github.com/vexxhost/atmosphere/inventory.yaml /home/zuul/src/github.com/vexxhost/atmosphere/molecule/csi/converge.yml 2026-02-25 16:38:46.472712 | instance | ERROR [csi > converge] Executed: Failed 2026-02-25 16:38:46.472938 | instance | ERROR Ansible return code was 2, command was: ansible-playbook --inventory /home/zuul/.ansible/tmp/molecule.v9Wo.csi/inventory --skip-tags molecule-notest,notest --inventory=/home/zuul/src/github.com/vexxhost/atmosphere/inventory.yaml /home/zuul/src/github.com/vexxhost/atmosphere/molecule/csi/converge.yml 2026-02-25 16:38:47.012749 | instance | ERROR 2026-02-25 16:38:47.013151 | instance | { 2026-02-25 16:38:47.013223 | instance | "delta": "0:01:57.405231", 2026-02-25 16:38:47.013271 | instance | "end": "2026-02-25 16:38:46.546707", 2026-02-25 16:38:47.013314 | instance | "msg": "non-zero return code", 2026-02-25 16:38:47.013352 | instance | "rc": 2, 2026-02-25 16:38:47.013382 | instance | "start": "2026-02-25 16:36:49.141476" 2026-02-25 16:38:47.013408 | instance | } failure 2026-02-25 16:38:47.021609 | 2026-02-25 16:38:47.021661 | PLAY RECAP 2026-02-25 16:38:47.021707 | instance | ok: 2 changed: 2 unreachable: 0 failed: 1 skipped: 0 rescued: 0 ignored: 0 2026-02-25 16:38:47.021729 | 2026-02-25 16:38:47.160201 | RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/run.yaml@main] 2026-02-25 16:38:47.164237 | POST-RUN START: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/post.yml@main] 2026-02-25 16:38:47.776503 | 2026-02-25 16:38:47.776651 | PLAY [all] 2026-02-25 16:38:47.792221 | 2026-02-25 16:38:47.792335 | TASK [gather-host-logs : creating directory for system status] 2026-02-25 16:38:48.146621 | instance | changed 2026-02-25 16:38:48.153763 | 2026-02-25 16:38:48.153847 | TASK [gather-host-logs : Get logs for each host] 2026-02-25 16:38:48.518501 | instance | + systemd-cgls --full --all --no-pager 2026-02-25 16:38:48.529910 | instance | + ip addr 2026-02-25 16:38:48.531712 | instance | + ip route 2026-02-25 16:38:48.533262 | instance | + lsblk 2026-02-25 16:38:48.538283 | instance | + mount 2026-02-25 16:38:48.539803 | instance | + docker images 2026-02-25 16:38:48.556996 | instance | + brctl show 2026-02-25 16:38:48.557388 | instance | /bin/bash: line 8: brctl: command not found 2026-02-25 16:38:48.557585 | instance | + ps aux --sort=-%mem 2026-02-25 16:38:48.571960 | instance | + dpkg -l 2026-02-25 16:38:48.582959 | instance | + CONTAINERS=($(docker ps -a --format '{{ .Names }}' --filter label=zuul)) 2026-02-25 16:38:48.583341 | instance | ++ docker ps -a --format '{{ .Names }}' --filter label=zuul 2026-02-25 16:38:48.597495 | instance | + '[' '!' -z '' ']' 2026-02-25 16:38:48.690627 | instance | ok: Runtime: 0:00:00.083435 2026-02-25 16:38:48.698585 | 2026-02-25 16:38:48.698661 | TASK [gather-host-logs : Downloads logs to executor] 2026-02-25 16:38:49.344329 | instance | changed: 2026-02-25 16:38:49.344531 | instance | created directory /var/lib/zuul/builds/b047b56ca6f64c5ca72689c14df52264/work/logs/instance 2026-02-25 16:38:49.344560 | instance | cd+++++++++ system/ 2026-02-25 16:38:49.344582 | instance | >f+++++++++ system/brctl-show.txt 2026-02-25 16:38:49.344603 | instance | >f+++++++++ system/docker-images.txt 2026-02-25 16:38:49.344623 | instance | >f+++++++++ system/ip-addr.txt 2026-02-25 16:38:49.344645 | instance | >f+++++++++ system/ip-route.txt 2026-02-25 16:38:49.344666 | instance | >f+++++++++ system/lsblk.txt 2026-02-25 16:38:49.344687 | instance | >f+++++++++ system/mount.txt 2026-02-25 16:38:49.344706 | instance | >f+++++++++ system/packages.txt 2026-02-25 16:38:49.344725 | instance | >f+++++++++ system/ps.txt 2026-02-25 16:38:49.344746 | instance | >f+++++++++ system/systemd-cgls.txt 2026-02-25 16:38:49.353734 | 2026-02-25 16:38:49.353801 | LOOP [helm-release-status : creating directory for helm release status] 2026-02-25 16:38:49.546274 | instance | changed: "values" 2026-02-25 16:38:49.706581 | instance | changed: "releases" 2026-02-25 16:38:49.719139 | 2026-02-25 16:38:49.719328 | TASK [helm-release-status : Gather get release status for helm charts] 2026-02-25 16:38:49.922689 | instance | /bin/bash: line 3: kubectl: command not found 2026-02-25 16:38:50.259507 | instance | ok: Runtime: 0:00:00.005326 2026-02-25 16:38:50.266591 | 2026-02-25 16:38:50.266665 | TASK [helm-release-status : Downloads logs to executor] 2026-02-25 16:38:50.739160 | instance | changed: 2026-02-25 16:38:50.739367 | instance | cd+++++++++ helm/ 2026-02-25 16:38:50.739395 | instance | cd+++++++++ helm/releases/ 2026-02-25 16:38:50.739416 | instance | cd+++++++++ helm/values/ 2026-02-25 16:38:50.748320 | 2026-02-25 16:38:50.748387 | TASK [describe-kubernetes-objects : creating directory for cluster scoped objects] 2026-02-25 16:38:50.945505 | instance | changed 2026-02-25 16:38:50.953755 | 2026-02-25 16:38:50.953853 | TASK [describe-kubernetes-objects : Gathering descriptions for cluster scoped objects] 2026-02-25 16:38:51.161963 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value 2026-02-25 16:38:51.162501 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value 2026-02-25 16:38:51.167561 | instance | environment: line 1: kubectl: command not found 2026-02-25 16:38:51.168262 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value 2026-02-25 16:38:51.169770 | instance | environment: line 1: kubectl: command not found 2026-02-25 16:38:51.170795 | instance | environment: line 1: kubectl: command not found 2026-02-25 16:38:51.171469 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value 2026-02-25 16:38:51.171506 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value 2026-02-25 16:38:51.173562 | instance | environment: line 1: kubectl: command not found 2026-02-25 16:38:51.174690 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value 2026-02-25 16:38:51.175016 | instance | environment: line 1: kubectl: command not found 2026-02-25 16:38:51.175998 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value 2026-02-25 16:38:51.493166 | instance | ok: Runtime: 0:00:00.022076 2026-02-25 16:38:51.500285 | 2026-02-25 16:38:51.500379 | TASK [describe-kubernetes-objects : creating directory for namespace scoped objects] 2026-02-25 16:38:51.687430 | instance | changed 2026-02-25 16:38:51.694409 | 2026-02-25 16:38:51.694498 | TASK [describe-kubernetes-objects : Gathering descriptions for namespace scoped objects] 2026-02-25 16:38:51.946355 | instance | environment: line 5: kubectl: command not found 2026-02-25 16:38:51.947229 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value 2026-02-25 16:38:51.947423 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value 2026-02-25 16:38:51.947440 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value 2026-02-25 16:38:52.280256 | instance | ok: Runtime: 0:00:00.006616 2026-02-25 16:38:52.290178 | 2026-02-25 16:38:52.290335 | TASK [describe-kubernetes-objects : Downloads logs to executor] 2026-02-25 16:38:52.802301 | instance | changed: 2026-02-25 16:38:52.802588 | instance | cd+++++++++ objects/ 2026-02-25 16:38:52.802616 | instance | cd+++++++++ objects/cluster/ 2026-02-25 16:38:52.802637 | instance | cd+++++++++ objects/namespaced/ 2026-02-25 16:38:52.812316 | 2026-02-25 16:38:52.812487 | TASK [gather-pod-logs : creating directory for pod logs] 2026-02-25 16:38:53.015536 | instance | changed 2026-02-25 16:38:53.021861 | 2026-02-25 16:38:53.021981 | TASK [gather-pod-logs : creating directory for failed pod logs] 2026-02-25 16:38:53.231980 | instance | changed 2026-02-25 16:38:53.238196 | 2026-02-25 16:38:53.238279 | TASK [gather-pod-logs : retrieve all kubernetes logs, current and previous (if they exist)] 2026-02-25 16:38:53.437375 | instance | environment: line 3: kubectl: command not found 2026-02-25 16:38:53.775041 | instance | ok: Runtime: 0:00:00.008760 2026-02-25 16:38:53.782426 | 2026-02-25 16:38:53.782519 | TASK [gather-pod-logs : Downloads pod logs to executor] 2026-02-25 16:38:54.259955 | instance | changed: 2026-02-25 16:38:54.260212 | instance | cd+++++++++ pod-logs/ 2026-02-25 16:38:54.260253 | instance | cd+++++++++ pod-logs/failed-pods/ 2026-02-25 16:38:54.270074 | 2026-02-25 16:38:54.270167 | TASK [gather-prom-metrics : creating directory for helm release descriptions] 2026-02-25 16:38:54.463572 | instance | changed 2026-02-25 16:38:54.469247 | 2026-02-25 16:38:54.469327 | TASK [gather-prom-metrics : Get metrics from exporter services in all namespaces] 2026-02-25 16:38:54.693044 | instance | /bin/bash: line 2: kubectl: command not found 2026-02-25 16:38:55.009250 | instance | ok: Runtime: 0:00:00.037662 2026-02-25 16:38:55.015165 | 2026-02-25 16:38:55.015264 | TASK [gather-prom-metrics : Get ceph metrics from ceph-mgr] 2026-02-25 16:38:55.231332 | instance | /bin/bash: line 2: kubectl: command not found 2026-02-25 16:38:55.260354 | instance | ceph-mgr endpoints: 2026-02-25 16:38:55.552871 | instance | ok: Runtime: 0:00:00.034867 2026-02-25 16:38:55.559931 | 2026-02-25 16:38:55.560009 | TASK [gather-prom-metrics : Get metrics from fluentd pods] 2026-02-25 16:38:55.763044 | instance | /bin/bash: line 4: kubectl: command not found 2026-02-25 16:38:56.096667 | instance | ok: Runtime: 0:00:00.032629 2026-02-25 16:38:56.103480 | 2026-02-25 16:38:56.103570 | TASK [gather-prom-metrics : Downloads logs to executor] 2026-02-25 16:38:56.591056 | instance | changed: cd+++++++++ prometheus/ 2026-02-25 16:38:56.602954 | 2026-02-25 16:38:56.603020 | TASK [gather-selenium-data : creating directory for helm release descriptions] 2026-02-25 16:38:56.801820 | instance | changed 2026-02-25 16:38:56.807670 | 2026-02-25 16:38:56.807740 | TASK [gather-selenium-data : Get selenium data] 2026-02-25 16:38:57.007086 | instance | + cp '/tmp/artifacts/*' /tmp/logs/selenium/. 2026-02-25 16:38:57.008107 | instance | cp: cannot stat '/tmp/artifacts/*': No such file or directory 2026-02-25 16:38:57.343809 | instance | ERROR 2026-02-25 16:38:57.344135 | instance | { 2026-02-25 16:38:57.344201 | instance | "delta": "0:00:00.004541", 2026-02-25 16:38:57.344249 | instance | "end": "2026-02-25 16:38:57.008354", 2026-02-25 16:38:57.344298 | instance | "msg": "non-zero return code", 2026-02-25 16:38:57.344341 | instance | "rc": 1, 2026-02-25 16:38:57.344382 | instance | "start": "2026-02-25 16:38:57.003813" 2026-02-25 16:38:57.344422 | instance | } 2026-02-25 16:38:57.344474 | instance | ERROR: Ignoring Errors 2026-02-25 16:38:57.353739 | 2026-02-25 16:38:57.354075 | TASK [gather-selenium-data : Downloads logs to executor] 2026-02-25 16:38:57.822373 | instance | changed: cd+++++++++ selenium/ 2026-02-25 16:38:57.829248 | 2026-02-25 16:38:57.829299 | PLAY RECAP 2026-02-25 16:38:57.829346 | instance | ok: 23 changed: 23 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 1 2026-02-25 16:38:57.829369 | 2026-02-25 16:38:57.947114 | POST-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/post.yml@main] 2026-02-25 16:38:57.949901 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main] 2026-02-25 16:38:58.543314 | 2026-02-25 16:38:58.543479 | PLAY [all] 2026-02-25 16:38:58.556480 | 2026-02-25 16:38:58.556601 | TASK [fetch-output : Set log path for multiple nodes] 2026-02-25 16:38:58.602816 | instance | skipping: Conditional result was False 2026-02-25 16:38:58.612043 | 2026-02-25 16:38:58.612176 | TASK [fetch-output : Set log path for single node] 2026-02-25 16:38:58.657613 | instance | ok 2026-02-25 16:38:58.662984 | 2026-02-25 16:38:58.663055 | LOOP [fetch-output : Ensure local output dirs] 2026-02-25 16:38:59.115179 | instance -> localhost | ok: "/var/lib/zuul/builds/b047b56ca6f64c5ca72689c14df52264/work/logs" 2026-02-25 16:38:59.340354 | instance -> localhost | changed: "/var/lib/zuul/builds/b047b56ca6f64c5ca72689c14df52264/work/artifacts" 2026-02-25 16:38:59.563006 | instance -> localhost | changed: "/var/lib/zuul/builds/b047b56ca6f64c5ca72689c14df52264/work/docs" 2026-02-25 16:38:59.576510 | 2026-02-25 16:38:59.576664 | LOOP [fetch-output : Collect logs, artifacts and docs] 2026-02-25 16:39:00.209425 | instance | changed: .d..t...... ./ 2026-02-25 16:39:00.209666 | instance | changed: All items complete 2026-02-25 16:39:00.209695 | 2026-02-25 16:39:00.676708 | instance | changed: .d..t...... ./ 2026-02-25 16:39:01.130784 | instance | changed: .d..t...... ./ 2026-02-25 16:39:01.157400 | 2026-02-25 16:39:01.157575 | LOOP [merge-output-to-logs : Move artifacts and docs to logs dir] 2026-02-25 16:39:01.594187 | instance -> localhost | ok: Item: artifacts Runtime: 0:00:00.009094 2026-02-25 16:39:01.842975 | instance -> localhost | ok: Item: docs Runtime: 0:00:00.008986 2026-02-25 16:39:01.854686 | 2026-02-25 16:39:01.854840 | PLAY [all] 2026-02-25 16:39:01.861236 | 2026-02-25 16:39:01.861305 | TASK [remove-build-sshkey : Remove the build SSH key from all nodes] 2026-02-25 16:39:02.280081 | instance | changed 2026-02-25 16:39:02.287331 | 2026-02-25 16:39:02.287387 | PLAY RECAP 2026-02-25 16:39:02.287439 | instance | ok: 5 changed: 4 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0 2026-02-25 16:39:02.287461 | 2026-02-25 16:39:02.429192 | POST-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main] 2026-02-25 16:39:02.433174 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post-logs.yaml@main] 2026-02-25 16:39:03.043441 | 2026-02-25 16:39:03.043570 | PLAY [localhost] 2026-02-25 16:39:03.053420 | 2026-02-25 16:39:03.053491 | TASK [Generate Zuul manifest] 2026-02-25 16:39:03.076769 | localhost | ok 2026-02-25 16:39:03.094529 | 2026-02-25 16:39:03.094616 | TASK [generate-zuul-manifest : Generate Zuul manifest] 2026-02-25 16:39:03.420371 | localhost | changed 2026-02-25 16:39:03.432884 | 2026-02-25 16:39:03.433000 | TASK [generate-zuul-manifest : Return Zuul manifest URL to Zuul] 2026-02-25 16:39:03.465177 | localhost | ok 2026-02-25 16:39:03.474023 | 2026-02-25 16:39:03.474138 | TASK [Upload logs] 2026-02-25 16:39:03.508417 | localhost | ok 2026-02-25 16:39:03.570458 | 2026-02-25 16:39:03.570624 | TASK [Set zuul-log-path fact] 2026-02-25 16:39:03.594049 | localhost | ok 2026-02-25 16:39:03.606031 | 2026-02-25 16:39:03.606190 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-02-25 16:39:03.639942 | localhost | ok 2026-02-25 16:39:03.650236 | 2026-02-25 16:39:03.650383 | TASK [upload-logs : Create log directories] 2026-02-25 16:39:04.088788 | localhost | changed 2026-02-25 16:39:04.094443 | 2026-02-25 16:39:04.094524 | TASK [upload-logs : Ensure logs are readable before uploading] 2026-02-25 16:39:04.514248 | localhost -> localhost | ok: Runtime: 0:00:00.006078 2026-02-25 16:39:04.521496 | 2026-02-25 16:39:04.521648 | TASK [upload-logs : Upload logs to log server] 2026-02-25 16:39:04.983796 | localhost | Output suppressed because no_log was given 2026-02-25 16:39:04.987413 | 2026-02-25 16:39:04.987482 | LOOP [upload-logs : Compress console log and json output] 2026-02-25 16:39:05.037140 | localhost | skipping: Conditional result was False 2026-02-25 16:39:05.044579 | localhost | skipping: Conditional result was False 2026-02-25 16:39:05.061647 | 2026-02-25 16:39:05.061858 | LOOP [upload-logs : Upload compressed console log and json output] 2026-02-25 16:39:05.109813 | localhost | skipping: Conditional result was False 2026-02-25 16:39:05.110341 | 2026-02-25 16:39:05.113646 | localhost | skipping: Conditional result was False 2026-02-25 16:39:05.127667 | 2026-02-25 16:39:05.127911 | LOOP [upload-logs : Upload console log and json output]