2026-04-08 04:07:54.398188 | Job console starting
2026-04-08 04:07:54.435554 | Updating git repos
2026-04-08 04:07:54.674296 | Cloning repos into workspace
2026-04-08 04:07:55.266898 | Restoring repo states
2026-04-08 04:07:55.284524 | Merging changes
2026-04-08 04:07:56.470680 | Checking out repos
2026-04-08 04:07:56.607954 | Preparing playbooks
2026-04-08 04:08:00.775010 | Running Ansible setup
2026-04-08 04:08:05.206677 | PRE-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main]
2026-04-08 04:08:05.807122 | 
2026-04-08 04:08:05.807240 | PLAY [localhost]
2026-04-08 04:08:05.814704 | 
2026-04-08 04:08:05.814773 | TASK [Gathering Facts]
2026-04-08 04:08:06.731386 | localhost | ok
2026-04-08 04:08:06.742295 | 
2026-04-08 04:08:06.742393 | TASK [Setup log path fact]
2026-04-08 04:08:06.763409 | localhost | ok
2026-04-08 04:08:06.777342 | 
2026-04-08 04:08:06.777406 | TASK [set-zuul-log-path-fact : Set log path for a build]
2026-04-08 04:08:06.807642 | localhost | ok
2026-04-08 04:08:06.816537 | 
2026-04-08 04:08:06.816620 | TASK [emit-job-header : Print job information]
2026-04-08 04:08:06.857834 | # Job Information
2026-04-08 04:08:06.858068 | Ansible Version: 2.16.16
2026-04-08 04:08:06.858139 | Job: atmosphere-molecule-aio-ovn
2026-04-08 04:08:06.858187 | Pipeline: check
2026-04-08 04:08:06.858231 | Executor: 0a8996d2b663
2026-04-08 04:08:06.858273 | Triggered by: https://github.com/vexxhost/atmosphere/pull/3809
2026-04-08 04:08:06.858323 | Event ID: 6dbdd510-3300-11f1-95e4-ee2cd8b60bfe
2026-04-08 04:08:06.862567 | 
2026-04-08 04:08:06.862632 | LOOP [emit-job-header : Print node information]
2026-04-08 04:08:06.957813 | localhost | ok:
2026-04-08 04:08:06.958048 | localhost | # Node Information
2026-04-08 04:08:06.958090 | localhost | Inventory Hostname: instance
2026-04-08 04:08:06.958121 | localhost | Hostname: np0000164194
2026-04-08 04:08:06.958150 | localhost | Username: zuul
2026-04-08 04:08:06.958183 | localhost | Distro: Ubuntu 22.04
2026-04-08 04:08:06.958212 | localhost | Provider: yul1
2026-04-08 04:08:06.958239 | localhost | Region: ca-ymq-1
2026-04-08 04:08:06.958266 | localhost | Label: ubuntu-jammy-16
2026-04-08 04:08:06.958291 | localhost | Product Name: OpenStack Nova
2026-04-08 04:08:06.958317 | localhost | Interface IP: 162.253.55.62
2026-04-08 04:08:06.970409 | 
2026-04-08 04:08:06.970505 | TASK [log-inventory : Ensure Zuul Ansible directory exists]
2026-04-08 04:08:07.372378 | localhost -> localhost | changed
2026-04-08 04:08:07.378762 | 
2026-04-08 04:08:07.378853 | TASK [log-inventory : Copy ansible inventory to logs dir]
2026-04-08 04:08:08.269077 | localhost -> localhost | changed
2026-04-08 04:08:08.278032 | 
2026-04-08 04:08:08.278119 | PLAY [all]
2026-04-08 04:08:08.288252 | 
2026-04-08 04:08:08.288321 | TASK [add-build-sshkey : Check to see if ssh key was already created for this build]
2026-04-08 04:08:08.519776 | instance -> localhost | ok
2026-04-08 04:08:08.527976 | 
2026-04-08 04:08:08.528071 | TASK [add-build-sshkey : Create a new key in workspace based on build UUID]
2026-04-08 04:08:08.564230 | instance | ok
2026-04-08 04:08:08.580083 | instance | included: /var/lib/zuul/builds/99af325baabc4fd49cb71d4bc4f2ea3f/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml
2026-04-08 04:08:08.585717 | 
2026-04-08 04:08:08.585777 | TASK [add-build-sshkey : Create Temp SSH key]
2026-04-08 04:08:10.119413 | instance -> localhost | Generating public/private rsa key pair.
2026-04-08 04:08:10.119679 | instance -> localhost | Your identification has been saved in /var/lib/zuul/builds/99af325baabc4fd49cb71d4bc4f2ea3f/work/99af325baabc4fd49cb71d4bc4f2ea3f_id_rsa
2026-04-08 04:08:10.119744 | instance -> localhost | Your public key has been saved in /var/lib/zuul/builds/99af325baabc4fd49cb71d4bc4f2ea3f/work/99af325baabc4fd49cb71d4bc4f2ea3f_id_rsa.pub
2026-04-08 04:08:10.119795 | instance -> localhost | The key fingerprint is:
2026-04-08 04:08:10.119843 | instance -> localhost | SHA256:m3SzSIpv42WxdRg4e07SlERF1jBeYfXCZo6OFsr9iSI zuul-build-sshkey
2026-04-08 04:08:10.119910 | instance -> localhost | The key's randomart image is:
2026-04-08 04:08:10.119959 | instance -> localhost | +---[RSA 3072]----+
2026-04-08 04:08:10.120017 | instance -> localhost | |         .oo*o+o.|
2026-04-08 04:08:10.120065 | instance -> localhost | |         o + =. .|
2026-04-08 04:08:10.120110 | instance -> localhost | |        o + . = .|
2026-04-08 04:08:10.120155 | instance -> localhost | |         = o = . |
2026-04-08 04:08:10.120199 | instance -> localhost | |        S X o .  |
2026-04-08 04:08:10.120244 | instance -> localhost | |     . = / B     |
2026-04-08 04:08:10.120288 | instance -> localhost | |    . . X * .    |
2026-04-08 04:08:10.120332 | instance -> localhost | |     .Eo.. o .   |
2026-04-08 04:08:10.120382 | instance -> localhost | |     ooo .. o    |
2026-04-08 04:08:10.120429 | instance -> localhost | +----[SHA256]-----+
2026-04-08 04:08:10.120524 | instance -> localhost | ok: Runtime: 0:00:01.146035
2026-04-08 04:08:10.133135 | 
2026-04-08 04:08:10.133200 | TASK [add-build-sshkey : Remote setup ssh keys (linux)]
2026-04-08 04:08:10.165896 | instance | ok
2026-04-08 04:08:10.176279 | instance | included: /var/lib/zuul/builds/99af325baabc4fd49cb71d4bc4f2ea3f/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml
2026-04-08 04:08:10.183729 | 
2026-04-08 04:08:10.183791 | TASK [add-build-sshkey : Remove previously added zuul-build-sshkey]
2026-04-08 04:08:10.208744 | instance | skipping: Conditional result was False
2026-04-08 04:08:10.218980 | 
2026-04-08 04:08:10.219065 | TASK [add-build-sshkey : Enable access via build key on all nodes]
2026-04-08 04:08:10.648817 | instance | changed
2026-04-08 04:08:10.653992 | 
2026-04-08 04:08:10.654053 | TASK [add-build-sshkey : Make sure user has a .ssh]
2026-04-08 04:08:10.834872 | instance | ok
2026-04-08 04:08:10.839793 | 
2026-04-08 04:08:10.839855 | TASK [add-build-sshkey : Install build private key as SSH key on all nodes]
2026-04-08 04:08:11.302091 | instance | changed
2026-04-08 04:08:11.309281 | 
2026-04-08 04:08:11.309361 | TASK [add-build-sshkey : Install build public key as SSH key on all nodes]
2026-04-08 04:08:11.765613 | instance | changed
2026-04-08 04:08:11.772563 | 
2026-04-08 04:08:11.772649 | TASK [add-build-sshkey : Remote setup ssh keys (windows)]
2026-04-08 04:08:11.798850 | instance | skipping: Conditional result was False
2026-04-08 04:08:11.810737 | 
2026-04-08 04:08:11.810851 | TASK [remove-zuul-sshkey : Remove master key from local agent]
2026-04-08 04:08:12.162175 | instance -> localhost | changed
2026-04-08 04:08:12.209822 | 
2026-04-08 04:08:12.209952 | TASK [add-build-sshkey : Add back temp key]
2026-04-08 04:08:12.486108 | instance -> localhost | Identity added: /var/lib/zuul/builds/99af325baabc4fd49cb71d4bc4f2ea3f/work/99af325baabc4fd49cb71d4bc4f2ea3f_id_rsa (zuul-build-sshkey)
2026-04-08 04:08:12.486276 | instance -> localhost | ok: Runtime: 0:00:00.013865
2026-04-08 04:08:12.491164 | 
2026-04-08 04:08:12.491226 | TASK [add-build-sshkey : Verify we can still SSH to all nodes]
2026-04-08 04:08:12.753818 | instance | ok
2026-04-08 04:08:12.761065 | 
2026-04-08 04:08:12.761157 | TASK [add-build-sshkey : Verify we can still SSH to all nodes (windows)]
2026-04-08 04:08:12.796212 | instance | skipping: Conditional result was False
2026-04-08 04:08:12.811777 | 
2026-04-08 04:08:12.811899 | TASK [prepare-workspace : Start zuul_console daemon.]
2026-04-08 04:08:13.093063 | instance | ok
2026-04-08 04:08:13.098754 | 
2026-04-08 04:08:13.098817 | TASK [prepare-workspace : Synchronize src repos to workspace directory.]
2026-04-08 04:08:14.608750 | instance | Output suppressed because no_log was given
2026-04-08 04:08:14.617879 | 
2026-04-08 04:08:14.617949 | LOOP [ensure-output-dirs : Empty Zuul Output directories by removing them]
2026-04-08 04:08:14.805183 | instance | ok: "logs"
2026-04-08 04:08:14.805393 | instance | ok: All items complete
2026-04-08 04:08:14.805420 | 
2026-04-08 04:08:14.966936 | instance | ok: "artifacts"
2026-04-08 04:08:15.115972 | instance | ok: "docs"
2026-04-08 04:08:15.126892 | 
2026-04-08 04:08:15.126980 | LOOP [ensure-output-dirs : Ensure Zuul Output directories exist]
2026-04-08 04:08:15.327594 | instance | changed: "logs"
2026-04-08 04:08:15.487676 | instance | changed: "artifacts"
2026-04-08 04:08:15.643318 | instance | changed: "docs"
2026-04-08 04:08:15.662875 | 
2026-04-08 04:08:15.662923 | PLAY RECAP
2026-04-08 04:08:15.662966 | instance | ok: 15 changed: 8 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0
2026-04-08 04:08:15.662994 | localhost | ok: 6 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0
2026-04-08 04:08:15.663016 | 
2026-04-08 04:08:15.774227 | PRE-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main]
2026-04-08 04:08:15.784777 | PRE-RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main]
2026-04-08 04:08:16.343065 | 
2026-04-08 04:08:16.343176 | PLAY [all]
2026-04-08 04:08:16.354624 | 
2026-04-08 04:08:16.354699 | TASK [setup-uv : Extract archive]
2026-04-08 04:08:18.537848 | instance | changed
2026-04-08 04:08:18.544101 | 
2026-04-08 04:08:18.544181 | TASK [setup-uv : Print version]
2026-04-08 04:08:19.207898 | instance | uv 0.8.13
2026-04-08 04:08:19.080656 | instance | ok: Runtime: 0:00:00.016491
2026-04-08 04:08:19.089083 | 
2026-04-08 04:08:19.089157 | PLAY RECAP
2026-04-08 04:08:19.089223 | instance | ok: 2 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0
2026-04-08 04:08:19.089256 | 
2026-04-08 04:08:19.206066 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main]
2026-04-08 04:08:19.214838 | PRE-RUN START: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/pre.yml@main]
2026-04-08 04:08:19.790715 | 
2026-04-08 04:08:19.790829 | PLAY [all]
2026-04-08 04:08:19.801940 | 
2026-04-08 04:08:19.802014 | TASK [Install "jq" for log collection]
2026-04-08 04:08:29.717611 | instance | changed
2026-04-08 04:08:29.720171 | 
2026-04-08 04:08:29.720251 | PLAY RECAP
2026-04-08 04:08:29.720319 | instance | ok: 1 changed: 1 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0
2026-04-08 04:08:29.720384 | 
2026-04-08 04:08:29.840429 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/pre.yml@main]
2026-04-08 04:08:29.847306 | RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/run.yaml@main]
2026-04-08 04:08:30.469507 | 
2026-04-08 04:08:30.469670 | PLAY [all]
2026-04-08 04:08:30.481729 | 
2026-04-08 04:08:30.481858 | TASK [Copy inventory file for Zuul]
2026-04-08 04:08:31.288271 | instance | changed
2026-04-08 04:08:31.295682 | 
2026-04-08 04:08:31.295788 | TASK [Switch "ansible_host" to private IP]
2026-04-08 04:08:31.604227 | instance | changed: 1 replacements made
2026-04-08 04:08:31.612134 | 
2026-04-08 04:08:31.612207 | TASK [Run Molecule scenario]
2026-04-08 04:08:32.071440 | instance | Using CPython 3.10.12 interpreter at: /usr/bin/python3
2026-04-08 04:08:32.071585 | instance | Creating virtual environment at: .venv
2026-04-08 04:08:32.096970 | instance |    Building atmosphere @ file:///home/zuul/src/github.com/vexxhost/atmosphere
2026-04-08 04:08:32.132297 | instance | Downloading openstacksdk (1.7MiB)
2026-04-08 04:08:32.132594 | instance | Downloading ansible-core (2.1MiB)
2026-04-08 04:08:32.132768 | instance | Downloading kubernetes (1.9MiB)
2026-04-08 04:08:32.133325 | instance | Downloading pydantic-core (2.0MiB)
2026-04-08 04:08:32.133495 | instance | Downloading cryptography (4.2MiB)
2026-04-08 04:08:32.133712 | instance | Downloading netaddr (2.2MiB)
2026-04-08 04:08:32.135003 | instance | Downloading setuptools (1.1MiB)
2026-04-08 04:08:32.135454 | instance | Downloading pygments (1.2MiB)
2026-04-08 04:08:32.407684 | instance | Downloading rjsonnet (1.2MiB)
2026-04-08 04:08:32.452077 | instance |    Building pyperclip==1.9.0
2026-04-08 04:08:32.580064 | instance |  Downloading pydantic-core
2026-04-08 04:08:32.597245 | instance |  Downloading rjsonnet
2026-04-08 04:08:32.624592 | instance |  Downloading netaddr
2026-04-08 04:08:32.640735 | instance |  Downloading pygments
2026-04-08 04:08:32.655957 | instance |  Downloading cryptography
2026-04-08 04:08:32.690517 | instance |  Downloading setuptools
2026-04-08 04:08:32.751406 | instance |  Downloading kubernetes
2026-04-08 04:08:32.786167 | instance |  Downloading ansible-core
2026-04-08 04:08:32.820030 | instance |  Downloading openstacksdk
2026-04-08 04:08:33.209354 | instance |       Built pyperclip==1.9.0
2026-04-08 04:08:33.343912 | instance |       Built atmosphere @ file:///home/zuul/src/github.com/vexxhost/atmosphere
2026-04-08 04:08:33.384856 | instance | Installed 83 packages in 39ms
2026-04-08 04:08:33.989176 | instance | WARNING  Molecule scenarios should migrate to 'extensions/molecule'
2026-04-08 04:08:34.606108 | instance | INFO     [aio > discovery] scenario test matrix: dependency, cleanup, destroy, syntax, create, prepare, converge, idempotence, side_effect, verify, cleanup, destroy
2026-04-08 04:08:34.606212 | instance | INFO     [aio > prerun] Performing prerun with role_name_check=0...
2026-04-08 04:09:54.196317 | instance | INFO     [aio > dependency] Executing
2026-04-08 04:09:54.196528 | instance | WARNING  [aio > dependency] Missing roles requirements file: requirements.yml
2026-04-08 04:09:54.196665 | instance | WARNING  [aio > dependency] Missing collections requirements file: collections.yml
2026-04-08 04:09:54.196846 | instance | WARNING  [aio > dependency] Executed: 2 missing (Remove from test_sequence to suppress)
2026-04-08 04:09:54.206652 | instance | INFO     [aio > cleanup] Executing
2026-04-08 04:09:54.207003 | instance | WARNING  [aio > cleanup] Executed: Missing playbook (Remove from test_sequence to suppress)
2026-04-08 04:09:54.216620 | instance | INFO     [aio > destroy] Executing
2026-04-08 04:09:54.216658 | instance | WARNING  [aio > destroy] Skipping, '--destroy=never' requested.
2026-04-08 04:09:54.216736 | instance | INFO     [aio > destroy] Executed: Successful
2026-04-08 04:09:54.226462 | instance | INFO     [aio > syntax] Executing
2026-04-08 04:09:57.039622 | instance |
2026-04-08 04:09:57.039749 | instance | playbook: /home/zuul/src/github.com/vexxhost/atmosphere/molecule/aio/converge.yml
2026-04-08 04:09:57.159873 | instance | INFO     [aio > syntax] Executed: Successful
2026-04-08 04:09:57.174310 | instance | INFO     [aio > create] Executing
2026-04-08 04:09:57.176163 | instance | WARNING  [aio > create] Executed: Missing playbook (Remove from test_sequence to suppress)
2026-04-08 04:09:57.185417 | instance | INFO     [aio > prepare] Executing
2026-04-08 04:09:58.024594 | instance |
2026-04-08 04:09:58.024862 | instance | PLAY [Prepare] *****************************************************************
2026-04-08 04:09:58.025115 | instance |
2026-04-08 04:09:58.025384 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 04:09:58.025653 | instance | Wednesday 08 April 2026  04:09:58 +0000 (0:00:00.031)       0:00:00.031 *******
2026-04-08 04:09:59.151562 | instance | [WARNING]: Platform linux on host instance is using the discovered Python
2026-04-08 04:09:59.151774 | instance | interpreter at /usr/bin/python3.10, but future installation of another Python
2026-04-08 04:09:59.152038 | instance | interpreter could change the meaning of that path. See
2026-04-08 04:09:59.152313 | instance | https://docs.ansible.com/ansible-
2026-04-08 04:09:59.152590 | instance | core/2.17/reference_appendices/interpreter_discovery.html for more information.
2026-04-08 04:09:59.163296 | instance | ok: [instance]
2026-04-08 04:09:59.163553 | instance |
2026-04-08 04:09:59.163829 | instance | TASK [Configure short hostname] ************************************************
2026-04-08 04:09:59.164104 | instance | Wednesday 08 April 2026  04:09:59 +0000 (0:00:01.139)       0:00:01.170 *******
2026-04-08 04:09:59.855793 | instance | changed: [instance]
2026-04-08 04:09:59.855893 | instance |
2026-04-08 04:09:59.856052 | instance | TASK [Ensure hostname inside hosts file] ***************************************
2026-04-08 04:09:59.856235 | instance | Wednesday 08 April 2026  04:09:59 +0000 (0:00:00.692)       0:00:01.863 *******
2026-04-08 04:10:00.117317 | instance | [WARNING]: Module remote_tmp /root/.ansible/tmp did not exist and was created
2026-04-08 04:10:00.117554 | instance | with a mode of 0700, this may cause issues when running as another user. To
2026-04-08 04:10:00.117839 | instance | avoid this, create the remote_tmp dir with the correct permissions manually
2026-04-08 04:10:00.127143 | instance | changed: [instance]
2026-04-08 04:10:00.127396 | instance |
2026-04-08 04:10:00.127673 | instance | TASK [Install "dirmngr" for GPG keyserver operations] **************************
2026-04-08 04:10:00.127957 | instance | Wednesday 08 April 2026  04:10:00 +0000 (0:00:00.271)       0:00:02.134 *******
2026-04-08 04:10:01.169533 | instance | ok: [instance]
2026-04-08 04:10:01.169755 | instance |
2026-04-08 04:10:01.170095 | instance | TASK [Purge "snapd" package] ***************************************************
2026-04-08 04:10:01.170367 | instance | Wednesday 08 April 2026  04:10:01 +0000 (0:00:01.042)       0:00:03.176 *******
2026-04-08 04:10:01.807610 | instance | ok: [instance]
2026-04-08 04:10:01.807807 | instance |
2026-04-08 04:10:01.808133 | instance | PLAY [Generate workspace for Atmosphere] ***************************************
2026-04-08 04:10:01.808381 | instance |
2026-04-08 04:10:01.808648 | instance | TASK [Create folders for workspace] ********************************************
2026-04-08 04:10:01.808917 | instance | Wednesday 08 April 2026  04:10:01 +0000 (0:00:00.637)       0:00:03.814 *******
2026-04-08 04:10:02.804819 | instance | changed: [localhost] => (item=group_vars)
2026-04-08 04:10:02.805050 | instance | changed: [localhost] => (item=group_vars/all)
2026-04-08 04:10:02.805348 | instance | changed: [localhost] => (item=group_vars/controllers)
2026-04-08 04:10:02.805644 | instance | changed: [localhost] => (item=group_vars/cephs)
2026-04-08 04:10:02.805990 | instance | changed: [localhost] => (item=group_vars/computes)
2026-04-08 04:10:02.806305 | instance | changed: [localhost] => (item=host_vars)
2026-04-08 04:10:02.806689 | instance |
2026-04-08 04:10:02.807117 | instance | PLAY [Generate Ceph control plane configuration for workspace] *****************
2026-04-08 04:10:02.807495 | instance |
2026-04-08 04:10:02.807833 | instance | TASK [Ensure the Ceph control plane configuration file exists] *****************
2026-04-08 04:10:02.808125 | instance | Wednesday 08 April 2026  04:10:02 +0000 (0:00:00.997)       0:00:04.812 *******
2026-04-08 04:10:02.974205 | instance | changed: [localhost]
2026-04-08 04:10:02.974444 | instance |
2026-04-08 04:10:02.974727 | instance | TASK [Load the current Ceph control plane configuration into a variable] *******
2026-04-08 04:10:02.974997 | instance | Wednesday 08 April 2026  04:10:02 +0000 (0:00:00.169)       0:00:04.982 *******
2026-04-08 04:10:03.010082 | instance | ok: [localhost]
2026-04-08 04:10:03.010312 | instance |
2026-04-08 04:10:03.010640 | instance | TASK [Generate Ceph control plane values for missing variables] ****************
2026-04-08 04:10:03.010931 | instance | Wednesday 08 April 2026  04:10:03 +0000 (0:00:00.035)       0:00:05.017 *******
2026-04-08 04:10:03.057670 | instance | ok: [localhost] => (item={'key': 'ceph_fsid', 'value': 'e0035e93-3a51-57e5-b9ee-b2bb6c5fa9a9'})
2026-04-08 04:10:03.057922 | instance | ok: [localhost] => (item={'key': 'ceph_mon_public_network', 'value': '10.96.240.0/24'})
2026-04-08 04:10:03.058167 | instance |
2026-04-08 04:10:03.058436 | instance | TASK [Write new Ceph control plane configuration file to disk] *****************
2026-04-08 04:10:03.058742 | instance | Wednesday 08 April 2026  04:10:03 +0000 (0:00:00.047)       0:00:05.065 *******
2026-04-08 04:10:03.586677 | instance | changed: [localhost]
2026-04-08 04:10:03.587019 | instance |
2026-04-08 04:10:03.587310 | instance | PLAY [Generate Ceph OSD configuration for workspace] ***************************
2026-04-08 04:10:03.587565 | instance |
2026-04-08 04:10:03.587842 | instance | TASK [Ensure the Ceph OSDs configuration file exists] **************************
2026-04-08 04:10:03.588072 | instance | Wednesday 08 April 2026  04:10:03 +0000 (0:00:00.528)       0:00:05.593 *******
2026-04-08 04:10:03.766570 | instance | changed: [localhost]
2026-04-08 04:10:03.766643 | instance |
2026-04-08 04:10:03.766883 | instance | TASK [Load the current Ceph OSDs configuration into a variable] ****************
2026-04-08 04:10:03.767013 | instance | Wednesday 08 April 2026  04:10:03 +0000 (0:00:00.179)       0:00:05.773 *******
2026-04-08 04:10:03.800410 | instance | ok: [localhost]
2026-04-08 04:10:03.800661 | instance |
2026-04-08 04:10:03.800945 | instance | TASK [Generate Ceph OSDs values for missing variables] *************************
2026-04-08 04:10:03.801223 | instance | Wednesday 08 April 2026  04:10:03 +0000 (0:00:00.034)       0:00:05.808 *******
2026-04-08 04:10:03.833486 | instance | ok: [localhost] => (item={'key': 'ceph_osd_devices', 'value': ['/dev/vdb', '/dev/vdc', '/dev/vdd']})
2026-04-08 04:10:03.833740 | instance |
2026-04-08 04:10:03.834008 | instance | TASK [Write new Ceph OSDs configuration file to disk] **************************
2026-04-08 04:10:03.834276 | instance | Wednesday 08 April 2026  04:10:03 +0000 (0:00:00.033)       0:00:05.841 *******
2026-04-08 04:10:04.193187 | instance | changed: [localhost]
2026-04-08 04:10:04.193262 | instance |
2026-04-08 04:10:04.193406 | instance | PLAY [Generate Kubernetes configuration for workspace] *************************
2026-04-08 04:10:04.193634 | instance |
2026-04-08 04:10:04.193761 | instance | TASK [Ensure the Kubernetes configuration file exists] *************************
2026-04-08 04:10:04.193979 | instance | Wednesday 08 April 2026  04:10:04 +0000 (0:00:00.359)       0:00:06.200 *******
2026-04-08 04:10:04.366713 | instance | changed: [localhost]
2026-04-08 04:10:04.366993 | instance |
2026-04-08 04:10:04.367276 | instance | TASK [Load the current Kubernetes configuration into a variable] ***************
2026-04-08 04:10:04.367547 | instance | Wednesday 08 April 2026  04:10:04 +0000 (0:00:00.173)       0:00:06.373 *******
2026-04-08 04:10:04.403643 | instance | ok: [localhost]
2026-04-08 04:10:04.403682 | instance |
2026-04-08 04:10:04.403874 | instance | TASK [Generate Kubernetes values for missing variables] ************************
2026-04-08 04:10:04.403939 | instance | Wednesday 08 April 2026  04:10:04 +0000 (0:00:00.037)       0:00:06.411 *******
2026-04-08 04:10:04.447666 | instance | ok: [localhost] => (item={'key': 'kubernetes_hostname', 'value': '10.96.240.10'})
2026-04-08 04:10:04.447724 | instance | ok: [localhost] => (item={'key': 'kubernetes_keepalived_vrid', 'value': 42})
2026-04-08 04:10:04.447915 | instance | ok: [localhost] => (item={'key': 'kubernetes_keepalived_vip', 'value': '10.96.240.10'})
2026-04-08 04:10:04.447974 | instance |
2026-04-08 04:10:04.448156 | instance | TASK [Write new Kubernetes configuration file to disk] *************************
2026-04-08 04:10:04.448215 | instance | Wednesday 08 April 2026  04:10:04 +0000 (0:00:00.043)       0:00:06.455 *******
2026-04-08 04:10:04.801329 | instance | changed: [localhost]
2026-04-08 04:10:04.801571 | instance |
2026-04-08 04:10:04.801908 | instance | PLAY [Generate Keepalived configuration for workspace] *************************
2026-04-08 04:10:04.802172 | instance |
2026-04-08 04:10:04.802478 | instance | TASK [Ensure the Keeaplived configuration file exists] *************************
2026-04-08 04:10:04.802857 | instance | Wednesday 08 April 2026  04:10:04 +0000 (0:00:00.353)       0:00:06.809 *******
2026-04-08 04:10:04.982850 | instance | changed: [localhost]
2026-04-08 04:10:04.983075 | instance |
2026-04-08 04:10:04.983349 | instance | TASK [Load the current Keepalived configuration into a variable] ***************
2026-04-08 04:10:04.983617 | instance | Wednesday 08 April 2026  04:10:04 +0000 (0:00:00.180)       0:00:06.989 *******
2026-04-08 04:10:05.018988 | instance | ok: [localhost]
2026-04-08 04:10:05.019256 | instance |
2026-04-08 04:10:05.019541 | instance | TASK [Generate Keepalived values for missing variables] ************************
2026-04-08 04:10:05.019823 | instance | Wednesday 08 April 2026  04:10:05 +0000 (0:00:00.036)       0:00:07.026 *******
2026-04-08 04:10:05.055018 | instance | ok: [localhost] => (item={'key': 'keepalived_interface', 'value': 'br-ex'})
2026-04-08 04:10:05.055297 | instance | ok: [localhost] => (item={'key': 'keepalived_vip', 'value': '10.96.250.10'})
2026-04-08 04:10:05.055554 | instance |
2026-04-08 04:10:05.055835 | instance | TASK [Write new Keepalived configuration file to disk] *************************
2026-04-08 04:10:05.056101 | instance | Wednesday 08 April 2026  04:10:05 +0000 (0:00:00.036)       0:00:07.062 *******
2026-04-08 04:10:05.389796 | instance | changed: [localhost]
2026-04-08 04:10:05.389881 | instance |
2026-04-08 04:10:05.390041 | instance | PLAY [Generate endpoints for workspace] ****************************************
2026-04-08 04:10:05.390180 | instance |
2026-04-08 04:10:05.390336 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 04:10:05.390594 | instance | Wednesday 08 April 2026  04:10:05 +0000 (0:00:00.334)       0:00:07.397 *******
2026-04-08 04:10:06.058535 | instance | ok: [localhost]
2026-04-08 04:10:06.058723 | instance |
2026-04-08 04:10:06.059052 | instance | TASK [Ensure the endpoints file exists] ****************************************
2026-04-08 04:10:06.059328 | instance | Wednesday 08 April 2026  04:10:06 +0000 (0:00:00.668)       0:00:08.065 *******
2026-04-08 04:10:06.242020 | instance | changed: [localhost]
2026-04-08 04:10:06.242278 | instance |
2026-04-08 04:10:06.242556 | instance | TASK [Load the current endpoints into a variable] ******************************
2026-04-08 04:10:06.242847 | instance | Wednesday 08 April 2026  04:10:06 +0000 (0:00:00.184)       0:00:08.249 *******
2026-04-08 04:10:06.279271 | instance | ok: [localhost]
2026-04-08 04:10:06.279582 | instance |
2026-04-08 04:10:06.279927 | instance | TASK [Generate endpoint skeleton for missing variables] ************************
2026-04-08 04:10:06.280200 | instance | Wednesday 08 April 2026  04:10:06 +0000 (0:00:00.037)       0:00:08.287 *******
2026-04-08 04:10:07.007074 | instance | ok: [localhost] => (item=keycloak_host)
2026-04-08 04:10:07.007286 | instance | ok: [localhost] => (item=kube_prometheus_stack_grafana_host)
2026-04-08 04:10:07.007593 | instance | ok: [localhost] => (item=kube_prometheus_stack_alertmanager_host)
2026-04-08 04:10:07.007857 | instance | ok: [localhost] => (item=kube_prometheus_stack_prometheus_host)
2026-04-08 04:10:07.008120 | instance | ok: [localhost] => (item=openstack_helm_endpoints_region_name)
2026-04-08 04:10:07.008351 | instance | ok: [localhost] => (item=openstack_helm_endpoints_keystone_api_host)
2026-04-08 04:10:07.008649 | instance | ok: [localhost] => (item=openstack_helm_endpoints_glance_api_host)
2026-04-08 04:10:07.008914 | instance | ok: [localhost] => (item=openstack_helm_endpoints_cinder_api_host)
2026-04-08 04:10:07.009177 | instance | ok: [localhost] => (item=openstack_helm_endpoints_placement_api_host)
2026-04-08 04:10:07.009494 | instance | ok: [localhost] => (item=openstack_helm_endpoints_barbican_api_host)
2026-04-08 04:10:07.009712 | instance | ok: [localhost] => (item=openstack_helm_endpoints_neutron_api_host)
2026-04-08 04:10:07.009974 | instance | ok: [localhost] => (item=openstack_helm_endpoints_nova_api_host)
2026-04-08 04:10:07.010237 | instance | ok: [localhost] => (item=openstack_helm_endpoints_nova_novnc_host)
2026-04-08 04:10:07.010507 | instance | ok: [localhost] => (item=openstack_helm_endpoints_ironic_api_host)
2026-04-08 04:10:07.010852 | instance | ok: [localhost] => (item=openstack_helm_endpoints_designate_api_host)
2026-04-08 04:10:07.011119 | instance | ok: [localhost] => (item=openstack_helm_endpoints_octavia_api_host)
2026-04-08 04:10:07.011382 | instance | ok: [localhost] => (item=openstack_helm_endpoints_magnum_api_host)
2026-04-08 04:10:07.011650 | instance | ok: [localhost] => (item=openstack_helm_endpoints_magnum_registry_host)
2026-04-08 04:10:07.011913 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_api_host)
2026-04-08 04:10:07.012175 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_cfn_api_host)
2026-04-08 04:10:07.012383 | instance | ok: [localhost] => (item=openstack_helm_endpoints_horizon_api_host)
2026-04-08 04:10:07.012657 | instance | ok: [localhost] => (item=openstack_helm_endpoints_rgw_host)
2026-04-08 04:10:07.012920 | instance | ok: [localhost] => (item=openstack_helm_endpoints_manila_api_host)
2026-04-08 04:10:07.013170 | instance |
2026-04-08 04:10:07.013455 | instance | TASK [Write new endpoints file to disk] ****************************************
2026-04-08 04:10:07.013725 | instance | Wednesday 08 April 2026  04:10:06 +0000 (0:00:00.726)       0:00:09.013 *******
2026-04-08 04:10:07.359087 | instance | changed: [localhost]
2026-04-08 04:10:07.359366 | instance |
2026-04-08 04:10:07.359682 | instance | TASK [Ensure the endpoints file exists] ****************************************
2026-04-08 04:10:07.359978 | instance | Wednesday 08 April 2026  04:10:07 +0000 (0:00:00.352)       0:00:09.366 *******
2026-04-08 04:10:07.533317 | instance | changed: [localhost]
2026-04-08 04:10:07.533600 | instance |
2026-04-08 04:10:07.533931 | instance | PLAY [Generate Neutron configuration for workspace] ****************************
2026-04-08 04:10:07.534237 | instance |
2026-04-08 04:10:07.534553 | instance | TASK [Ensure the Neutron configuration file exists] ****************************
2026-04-08 04:10:07.534898 | instance | Wednesday 08 April 2026  04:10:07 +0000 (0:00:00.174)       0:00:09.540 *******
2026-04-08 04:10:07.727465 | instance | changed: [localhost]
2026-04-08 04:10:07.727587 | instance |
2026-04-08 04:10:07.727885 | instance | TASK [Load the current Neutron configuration into a variable] ******************
2026-04-08 04:10:07.728158 | instance | Wednesday 08 April 2026  04:10:07 +0000 (0:00:00.193)       0:00:09.734 *******
2026-04-08 04:10:07.766356 | instance | ok: [localhost]
2026-04-08 04:10:07.766469 | instance |
2026-04-08 04:10:07.766808 | instance | TASK [Generate Neutron values for missing variables] ***************************
2026-04-08 04:10:07.767087 | instance | Wednesday 08 April 2026  04:10:07 +0000 (0:00:00.038)       0:00:09.773 *******
2026-04-08 04:10:07.809213 | instance | ok: [localhost] => (item={'key': 'neutron_networks', 'value': [{'name': 'public', 'external': True, 'shared': True, 'mtu_size': 1500, 'port_security_enabled': True, 'provider_network_type': 'flat', 'provider_physical_network': 'external', 'subnets': [{'name': 'public-subnet', 'cidr': '10.96.250.0/24', 'gateway_ip': '10.96.250.10', 'allocation_pool_start': '10.96.250.200', 'allocation_pool_end': '10.96.250.220', 'enable_dhcp': True}]}]})
2026-04-08 04:10:07.809326 | instance |
2026-04-08 04:10:07.809633 | instance | TASK [Write new Neutron configuration file to disk] ****************************
2026-04-08 04:10:07.809911 | instance | Wednesday 08 April 2026  04:10:07 +0000 (0:00:00.043)       0:00:09.816 *******
2026-04-08 04:10:08.157665 | instance | changed: [localhost]
2026-04-08 04:10:08.157800 | instance |
2026-04-08 04:10:08.157851 | instance | PLAY [Generate Nova configuration for workspace] *******************************
2026-04-08 04:10:08.158025 | instance |
2026-04-08 04:10:08.158218 | instance | TASK [Ensure the Nova configuration file exists] *******************************
2026-04-08 04:10:08.158382 | instance | Wednesday 08 April 2026  04:10:08 +0000 (0:00:00.348)       0:00:10.165 *******
2026-04-08 04:10:08.335976 | instance | changed: [localhost]
2026-04-08 04:10:08.336079 | instance |
2026-04-08 04:10:08.336240 | instance | TASK [Load the current Nova configuration into a variable] *********************
2026-04-08 04:10:08.336422 | instance | Wednesday 08 April 2026  04:10:08 +0000 (0:00:00.178)       0:00:10.343 *******
2026-04-08 04:10:08.378673 | instance | ok: [localhost]
2026-04-08 04:10:08.378770 | instance |
2026-04-08 04:10:08.378929 | instance | TASK [Generate Nova values for missing variables] ******************************
2026-04-08 04:10:08.379153 | instance | Wednesday 08 April 2026  04:10:08 +0000 (0:00:00.042)       0:00:10.386 *******
2026-04-08 04:10:08.424064 | instance | ok: [localhost] => (item={'key': 'nova_flavors', 'value': [{'name': 'm1.tiny', 'ram': 512, 'disk': 1, 'vcpus': 1}, {'name': 'm1.small', 'ram': 2048, 'disk': 20, 'vcpus': 1}, {'name': 'm1.medium', 'ram': 4096, 'disk': 40, 'vcpus': 2}, {'name': 'm1.large', 'ram': 8192, 'disk': 80, 'vcpus': 4}, {'name': 'm1.xlarge', 'ram': 16384, 'disk': 160, 'vcpus': 8}]})
2026-04-08 04:10:08.424202 | instance |
2026-04-08 04:10:08.424493 | instance | TASK [Write new Nova configuration file to disk] *******************************
2026-04-08 04:10:08.424652 | instance | Wednesday 08 April 2026  04:10:08 +0000 (0:00:00.045)       0:00:10.431 *******
2026-04-08 04:10:08.768034 | instance | changed: [localhost]
2026-04-08 04:10:08.768286 | instance |
2026-04-08 04:10:08.768913 | instance | PLAY [Generate secrets for workspace] ******************************************
2026-04-08 04:10:08.769245 | instance |
2026-04-08 04:10:08.769549 | instance | TASK [Ensure the secrets file exists] ******************************************
2026-04-08 04:10:08.769910 | instance | Wednesday 08 April 2026  04:10:08 +0000 (0:00:00.343)       0:00:10.775 *******
2026-04-08 04:10:08.949502 | instance | changed: [localhost]
2026-04-08 04:10:08.949775 | instance |
2026-04-08 04:10:08.950075 | instance | TASK [Load the current secrets into a variable] ********************************
2026-04-08 04:10:08.950362 | instance | Wednesday 08 April 2026  04:10:08 +0000 (0:00:00.181)       0:00:10.956 *******
2026-04-08 04:10:08.988097 | instance | ok: [localhost]
2026-04-08 04:10:08.988386 | instance |
2026-04-08 04:10:08.988693 | instance | TASK [Generate secrets for missing variables] **********************************
2026-04-08 04:10:08.989011 | instance | Wednesday 08 April 2026  04:10:08 +0000 (0:00:00.038)       0:00:10.995 *******
2026-04-08 04:10:09.385661 | instance | ok: [localhost] => (item=heat_auth_encryption_key)
2026-04-08 04:10:09.386073 | instance | ok: [localhost] => (item=keepalived_password)
2026-04-08 04:10:09.386421 | instance | ok: [localhost] => (item=keycloak_admin_password)
2026-04-08 04:10:09.386797 | instance | ok: [localhost] => (item=keycloak_database_password)
2026-04-08 04:10:09.387135 | instance | ok: [localhost] => (item=keystone_keycloak_client_secret)
2026-04-08 04:10:09.387470 | instance | ok: [localhost] => (item=keystone_oidc_crypto_passphrase)
2026-04-08 04:10:09.387806 | instance | ok: [localhost] => (item=kube_prometheus_stack_grafana_admin_password)
2026-04-08 04:10:09.388138 | instance | ok: [localhost] => (item=octavia_heartbeat_key)
2026-04-08 04:10:09.388476 | instance | ok: [localhost] => (item=openstack_helm_endpoints_rabbitmq_admin_password)
2026-04-08 04:10:09.388809 | instance | ok: [localhost] => (item=openstack_helm_endpoints_memcached_secret_key)
2026-04-08 04:10:09.389163 | instance | ok: [localhost] => (item=openstack_helm_endpoints_keystone_admin_password)
2026-04-08 04:10:09.389497 | instance | ok: [localhost] => (item=openstack_helm_endpoints_keystone_mariadb_password)
2026-04-08 04:10:09.389832 | instance | ok: [localhost] => (item=openstack_helm_endpoints_keystone_rabbitmq_password)
2026-04-08 04:10:09.390165 | instance | ok: [localhost] => (item=openstack_helm_endpoints_glance_keystone_password)
2026-04-08 04:10:09.390533 | instance | ok: [localhost] => (item=openstack_helm_endpoints_glance_mariadb_password)
2026-04-08 04:10:09.390883 | instance | ok: [localhost] => (item=openstack_helm_endpoints_glance_rabbitmq_password)
2026-04-08 04:10:09.391217 | instance | ok: [localhost] => (item=openstack_helm_endpoints_cinder_keystone_password)
2026-04-08 04:10:09.391551 | instance | ok: [localhost] => (item=openstack_helm_endpoints_cinder_mariadb_password)
2026-04-08 04:10:09.391882 | instance | ok: [localhost] => (item=openstack_helm_endpoints_cinder_rabbitmq_password)
2026-04-08 04:10:09.392212 | instance | ok: [localhost] => (item=openstack_helm_endpoints_placement_keystone_password)
2026-04-08 04:10:09.392545 | instance | ok: [localhost] => (item=openstack_helm_endpoints_placement_mariadb_password)
2026-04-08 04:10:09.392876 | instance | ok: [localhost] => (item=openstack_helm_endpoints_barbican_keystone_password)
2026-04-08 04:10:09.393259 | instance | ok: [localhost] => (item=openstack_helm_endpoints_barbican_mariadb_password)
2026-04-08 04:10:09.393578 | instance | ok: [localhost] => (item=openstack_helm_endpoints_neutron_keystone_password)
2026-04-08 04:10:09.393726 | instance | ok: [localhost] => (item=openstack_helm_endpoints_neutron_mariadb_password)
2026-04-08 04:10:09.393876 | instance | ok: [localhost] => (item=openstack_helm_endpoints_neutron_rabbitmq_password)
2026-04-08 04:10:09.394024 | instance | ok: [localhost] => (item=openstack_helm_endpoints_neutron_metadata_secret)
2026-04-08 04:10:09.394174 | instance | ok: [localhost] => (item=openstack_helm_endpoints_nova_keystone_password)
2026-04-08 04:10:09.394323 | instance | ok: [localhost] => (item=openstack_helm_endpoints_nova_mariadb_password)
2026-04-08 04:10:09.394472 | instance | ok: [localhost] => (item=openstack_helm_endpoints_nova_rabbitmq_password)
2026-04-08 04:10:09.394636 | instance | ok: [localhost] => (item=openstack_helm_endpoints_ironic_keystone_password)
2026-04-08 04:10:09.394786 | instance | ok: [localhost] => (item=openstack_helm_endpoints_ironic_mariadb_password)
2026-04-08 04:10:09.394935 | instance | ok: [localhost] => (item=openstack_helm_endpoints_ironic_rabbitmq_password)
2026-04-08 04:10:09.395085 | instance | ok: [localhost] => (item=openstack_helm_endpoints_designate_keystone_password)
2026-04-08 04:10:09.395234 | instance | ok: [localhost] => (item=openstack_helm_endpoints_designate_mariadb_password)
2026-04-08 04:10:09.395382 | instance | ok: [localhost] => (item=openstack_helm_endpoints_designate_rabbitmq_password)
2026-04-08 04:10:09.395550 | instance | ok: [localhost] => (item=openstack_helm_endpoints_octavia_keystone_password)
2026-04-08 04:10:09.395698 | instance | ok: [localhost] => (item=openstack_helm_endpoints_octavia_mariadb_password)
2026-04-08 04:10:09.395847 | instance | ok: [localhost] => (item=openstack_helm_endpoints_octavia_rabbitmq_password)
2026-04-08 04:10:09.395996 | instance | ok: [localhost] => (item=openstack_helm_endpoints_magnum_keystone_password)
2026-04-08 04:10:09.396147 | instance | ok: [localhost] => (item=openstack_helm_endpoints_magnum_mariadb_password)
2026-04-08 04:10:09.396297 | instance | ok: [localhost] => (item=openstack_helm_endpoints_magnum_rabbitmq_password)
2026-04-08 04:10:09.396446 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_keystone_password)
2026-04-08 04:10:09.396611 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_trustee_keystone_password)
2026-04-08 04:10:09.396767 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_stack_user_keystone_password)
2026-04-08 04:10:09.396928 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_mariadb_password)
2026-04-08 04:10:09.397079 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_rabbitmq_password)
2026-04-08 04:10:09.397229 | instance | ok: [localhost] => (item=openstack_helm_endpoints_horizon_mariadb_password)
2026-04-08 04:10:09.397378 | instance | ok: [localhost] => (item=openstack_helm_endpoints_tempest_keystone_password)
2026-04-08 04:10:09.397531 | instance | ok: [localhost] => (item=openstack_helm_endpoints_openstack_exporter_keystone_password)
2026-04-08 04:10:09.397681 | instance | ok: [localhost] => (item=openstack_helm_endpoints_rgw_keystone_password)
2026-04-08 04:10:09.397830 | instance | ok: [localhost] => (item=openstack_helm_endpoints_manila_keystone_password)
2026-04-08 04:10:09.397979 | instance | ok: [localhost] => (item=openstack_helm_endpoints_manila_mariadb_password)
2026-04-08 04:10:09.398135 | instance | ok: [localhost] => (item=openstack_helm_endpoints_staffeln_mariadb_password)
2026-04-08 04:10:09.398279 | instance |
2026-04-08 04:10:09.398429 | instance | TASK [Generate base64 encoded secrets] *****************************************
2026-04-08 04:10:09.398597 | instance | Wednesday 08 April 2026  04:10:09 +0000 (0:00:00.397)       0:00:11.393 *******
2026-04-08 04:10:09.443009 | instance | ok: [localhost] => (item=barbican_kek)
2026-04-08 04:10:09.443145 | instance |
2026-04-08 04:10:09.443346 | instance | TASK [Generate temporary files for generating keys for missing variables] ******
2026-04-08 04:10:09.443503 | instance | Wednesday 08 April 2026  04:10:09 +0000 (0:00:00.056)       0:00:11.449 *******
2026-04-08 04:10:09.843208 | instance | changed: [localhost] => (item=manila_ssh_key)
2026-04-08 04:10:09.843480 | instance | changed: [localhost] => (item=nova_ssh_key)
2026-04-08 04:10:09.843672 | instance |
2026-04-08 04:10:09.843875 | instance | TASK [Generate SSH keys for missing variables] *********************************
2026-04-08 04:10:09.844075 | instance | Wednesday 08 April 2026  04:10:09 +0000 (0:00:00.400)       0:00:11.850 *******
2026-04-08 04:10:13.686025 | instance | changed: [localhost] => (item=manila_ssh_key)
2026-04-08 04:10:13.686113 | instance | changed: [localhost] => (item=nova_ssh_key)
2026-04-08 04:10:13.686607 | instance |
2026-04-08 04:10:13.686683 | instance | TASK [Set values for SSH keys] *************************************************
2026-04-08 04:10:13.686701 | instance | Wednesday 08 April 2026  04:10:13 +0000 (0:00:03.843)       0:00:15.693 *******
2026-04-08 04:10:13.737066 | instance | ok: [localhost] => (item=manila_ssh_key)
2026-04-08 04:10:13.737948 | instance | ok: [localhost] => (item=nova_ssh_key)
2026-04-08 04:10:13.738005 | instance |
2026-04-08 04:10:13.738017 | instance | TASK [Delete the temporary files generated for SSH keys] ***********************
2026-04-08 04:10:13.738028 | instance | Wednesday 08 April 2026  04:10:13 +0000 (0:00:00.051)       0:00:15.744 *******
2026-04-08 04:10:14.089783 | instance | changed: [localhost] => (item=manila_ssh_key)
2026-04-08 04:10:14.089846 | instance | changed: [localhost] => (item=nova_ssh_key)
2026-04-08 04:10:14.089859 | instance |
2026-04-08 04:10:14.089869 | instance | TASK [Write new secrets file to disk] ******************************************
2026-04-08 04:10:14.089880 | instance | Wednesday 08 April 2026  04:10:14 +0000 (0:00:00.350)       0:00:16.094 *******
2026-04-08 04:10:14.451954 | instance | changed: [localhost]
2026-04-08 04:10:14.452049 | instance |
2026-04-08 04:10:14.452061 | instance | TASK [Encrypt secrets file with Vault password] ********************************
2026-04-08 04:10:14.452072 | instance | Wednesday 08 April 2026  04:10:14 +0000 (0:00:00.364)       0:00:16.459 *******
2026-04-08 04:10:14.485460 | instance | skipping: [localhost]
2026-04-08 04:10:14.485582 | instance |
2026-04-08 04:10:14.486291 | instance | PLAY [Overwrite OSD devices with LVM-backed paths] *****************************
2026-04-08 04:10:14.486333 | instance |
2026-04-08 04:10:14.486341 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 04:10:14.486348 | instance | Wednesday 08 April 2026  04:10:14 +0000 (0:00:00.034)       0:00:16.493 *******
2026-04-08 04:10:15.184165 | instance | ok: [instance]
2026-04-08 04:10:15.184251 | instance |
2026-04-08 04:10:15.184364 | instance | TASK [Overwrite existing osds.yml file] ****************************************
2026-04-08 04:10:15.184500 | instance | Wednesday 08 April 2026  04:10:15 +0000 (0:00:00.695)       0:00:17.189 *******
2026-04-08 04:10:15.659885 | instance | changed: [instance]
2026-04-08 04:10:15.660104 | instance |
2026-04-08 04:10:15.660427 | instance | PLAY [Setup networking] ********************************************************
2026-04-08 04:10:15.660694 | instance |
2026-04-08 04:10:15.660964 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 04:10:15.661230 | instance | Wednesday 08 April 2026  04:10:15 +0000 (0:00:00.477)       0:00:17.667 *******
2026-04-08 04:10:16.356066 | instance | ok: [instance]
2026-04-08 04:10:16.356426 | instance |
2026-04-08 04:10:16.356760 | instance | TASK [Create bridge for management network] ************************************
2026-04-08 04:10:16.357007 | instance | Wednesday 08 April 2026  04:10:16 +0000 (0:00:00.696)       0:00:18.363 *******
2026-04-08 04:10:16.697910 | instance | ok: [instance]
2026-04-08 04:10:16.697982 | instance |
2026-04-08 04:10:16.698108 | instance | TASK [Create fake interface for management bridge] *****************************
2026-04-08 04:10:16.698232 | instance | Wednesday 08 April 2026  04:10:16 +0000 (0:00:00.342)       0:00:18.705 *******
2026-04-08 04:10:16.916053 | instance | ok: [instance]
2026-04-08 04:10:16.916342 | instance |
2026-04-08 04:10:16.916721 | instance | TASK [Assign dummy interface to management bridge] *****************************
2026-04-08 04:10:16.916958 | instance | Wednesday 08 April 2026  04:10:16 +0000 (0:00:00.217)       0:00:18.923 *******
2026-04-08 04:10:17.118677 | instance | ok: [instance]
2026-04-08 04:10:17.118909 | instance |
2026-04-08 04:10:17.119190 | instance | TASK [Assign IP address for management bridge] *********************************
2026-04-08 04:10:17.119468 | instance | Wednesday 08 April 2026  04:10:17 +0000 (0:00:00.203)       0:00:19.126 *******
2026-04-08 04:10:17.316994 | instance | ok: [instance]
2026-04-08 04:10:17.317210 | instance |
2026-04-08 04:10:17.317478 | instance | TASK [Bring up interfaces] *****************************************************
2026-04-08 04:10:17.317764 | instance | Wednesday 08 April 2026  04:10:17 +0000 (0:00:00.198)       0:00:19.324 *******
2026-04-08 04:10:17.696894 | instance | ok: [instance] => (item=br-mgmt)
2026-04-08 04:10:17.697321 | instance | ok: [instance] => (item=dummy0)
2026-04-08 04:10:17.697654 | instance |
2026-04-08 04:10:17.698008 | instance | PLAY [Create devices for Ceph] *************************************************
2026-04-08 04:10:17.698352 | instance |
2026-04-08 04:10:17.698727 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 04:10:17.699066 | instance | Wednesday 08 April 2026  04:10:17 +0000 (0:00:00.379)       0:00:19.704 *******
2026-04-08 04:10:18.447305 | instance | ok: [instance]
2026-04-08 04:10:18.447370 | instance |
2026-04-08 04:10:18.447429 | instance | TASK [Install depedencies] *****************************************************
2026-04-08 04:10:18.447597 | instance | Wednesday 08 April 2026  04:10:18 +0000 (0:00:00.748)       0:00:20.452 *******
2026-04-08 04:10:37.898383 | instance | changed: [instance]
2026-04-08 04:10:37.898437 | instance |
2026-04-08 04:10:37.898443 | instance | TASK [Start up service] ********************************************************
2026-04-08 04:10:37.898455 | instance | Wednesday 08 April 2026  04:10:37 +0000 (0:00:19.451)       0:00:39.904 *******
2026-04-08 04:10:38.469978 | instance | ok: [instance]
2026-04-08 04:10:38.470061 | instance |
2026-04-08 04:10:38.470070 | instance | TASK [Generate lvm.conf] *******************************************************
2026-04-08 04:10:38.470076 | instance | Wednesday 08 April 2026  04:10:38 +0000 (0:00:00.572)       0:00:40.477 *******
2026-04-08 04:10:38.692205 | instance | ok: [instance]
2026-04-08 04:10:38.692273 | instance |
2026-04-08 04:10:38.692281 | instance | TASK [Write /etc/lvm/lvm.conf] *************************************************
2026-04-08 04:10:38.692288 | instance | Wednesday 08 April 2026  04:10:38 +0000 (0:00:00.222)       0:00:40.699 *******
2026-04-08 04:10:39.177671 | instance | changed: [instance]
2026-04-08 04:10:39.177731 | instance |
2026-04-08 04:10:39.177737 | instance | TASK [Get list of all loopback devices] ****************************************
2026-04-08 04:10:39.177742 | instance | Wednesday 08 April 2026  04:10:39 +0000 (0:00:00.485)       0:00:41.185 *******
2026-04-08 04:10:39.387727 | instance | ok: [instance]
2026-04-08 04:10:39.387772 | instance |
2026-04-08 04:10:39.387778 | instance | TASK [Fail if there is any existing loopback devices] **************************
2026-04-08 04:10:39.387783 | instance | Wednesday 08 April 2026  04:10:39 +0000 (0:00:00.209)       0:00:41.394 *******
2026-04-08 04:10:39.411260 | instance | skipping: [instance]
2026-04-08 04:10:39.411283 | instance |
2026-04-08 04:10:39.411290 | instance | TASK [Create devices for Ceph] *************************************************
2026-04-08 04:10:39.411296 | instance | Wednesday 08 April 2026  04:10:39 +0000 (0:00:00.024)       0:00:41.418 *******
2026-04-08 04:10:39.962611 | instance | changed: [instance] => (item=osd0)
2026-04-08 04:10:39.962696 | instance | changed: [instance] => (item=osd1)
2026-04-08 04:10:39.962716 | instance | changed: [instance] => (item=osd2)
2026-04-08 04:10:39.963200 | instance |
2026-04-08 04:10:39.963271 | instance | TASK [Set permissions on loopback devices] *************************************
2026-04-08 04:10:39.963279 | instance | Wednesday 08 April 2026  04:10:39 +0000 (0:00:00.550)       0:00:41.969 *******
2026-04-08 04:10:40.478919 | instance | changed: [instance] => (item=osd0)
2026-04-08 04:10:40.479042 | instance | changed: [instance] => (item=osd1)
2026-04-08 04:10:40.479155 | instance | changed: [instance] => (item=osd2)
2026-04-08 04:10:40.479404 | instance |
2026-04-08 04:10:40.479517 | instance | TASK [Start loop devices] ******************************************************
2026-04-08 04:10:40.479693 | instance | Wednesday 08 April 2026  04:10:40 +0000 (0:00:00.516)       0:00:42.486 *******
2026-04-08 04:10:41.219856 | instance | changed: [instance] => (item=osd0)
2026-04-08 04:10:41.220078 | instance | changed: [instance] => (item=osd1)
2026-04-08 04:10:41.220335 | instance | changed: [instance] => (item=osd2)
2026-04-08 04:10:41.220603 | instance |
2026-04-08 04:10:41.220850 | instance | TASK [Create a volume group for each loop device] ******************************
2026-04-08 04:10:41.221135 | instance | Wednesday 08 April 2026  04:10:41 +0000 (0:00:00.740)       0:00:43.227 *******
2026-04-08 04:10:44.365315 | instance | changed: [instance] => (item=osd0)
2026-04-08 04:10:44.365420 | instance | changed: [instance] => (item=osd1)
2026-04-08 04:10:44.365575 | instance | changed: [instance] => (item=osd2)
2026-04-08 04:10:44.365736 | instance |
2026-04-08 04:10:44.365929 | instance | TASK [Create a logical volume for each loop device] ****************************
2026-04-08 04:10:44.366091 | instance | Wednesday 08 April 2026  04:10:44 +0000 (0:00:03.145)       0:00:46.372 *******
2026-04-08 04:10:46.284858 | instance | changed: [instance] => (item=ceph-instance-osd0)
2026-04-08 04:10:46.284917 | instance | changed: [instance] => (item=ceph-instance-osd1)
2026-04-08 04:10:46.284922 | instance | changed: [instance] => (item=ceph-instance-osd2)
2026-04-08 04:10:46.284926 | instance |
2026-04-08 04:10:46.284931 | instance | PLAY [controllers] *************************************************************
2026-04-08 04:10:46.284936 | instance |
2026-04-08 04:10:46.284940 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 04:10:46.284944 | instance | Wednesday 08 April 2026  04:10:46 +0000 (0:00:01.918)       0:00:48.291 *******
2026-04-08 04:10:47.185545 | instance | ok: [instance]
2026-04-08 04:10:47.185666 | instance |
2026-04-08 04:10:47.186042 | instance | TASK [Set masquerade rule] *****************************************************
2026-04-08 04:10:47.186114 | instance | Wednesday 08 April 2026  04:10:47 +0000 (0:00:00.901)       0:00:49.192 *******
2026-04-08 04:10:47.677123 | instance | changed: [instance]
2026-04-08 04:10:47.677346 | instance |
2026-04-08 04:10:47.677793 | instance | PLAY RECAP *********************************************************************
2026-04-08 04:10:47.678043 | instance | instance                   : ok=26   changed=11   unreachable=0    failed=0    skipped=1    rescued=0    ignored=0
2026-04-08 04:10:47.678270 | instance | localhost                  : ok=40   changed=21   unreachable=0    failed=0    skipped=1    rescued=0    ignored=0
2026-04-08 04:10:47.678468 | instance |
2026-04-08 04:10:47.678697 | instance | Wednesday 08 April 2026  04:10:47 +0000 (0:00:00.491)       0:00:49.684 *******
2026-04-08 04:10:47.678908 | instance | ===============================================================================
2026-04-08 04:10:47.679112 | instance | Install depedencies ---------------------------------------------------- 19.45s
2026-04-08 04:10:47.679316 | instance | Generate SSH keys for missing variables --------------------------------- 3.84s
2026-04-08 04:10:47.679520 | instance | Create a volume group for each loop device ------------------------------ 3.15s
2026-04-08 04:10:47.679723 | instance | Create a logical volume for each loop device ---------------------------- 1.92s
2026-04-08 04:10:47.679924 | instance | Gathering Facts --------------------------------------------------------- 1.14s
2026-04-08 04:10:47.680128 | instance | Install "dirmngr" for GPG keyserver operations -------------------------- 1.04s
2026-04-08 04:10:47.680330 | instance | Create folders for workspace -------------------------------------------- 1.00s
2026-04-08 04:10:47.680530 | instance | Gathering Facts --------------------------------------------------------- 0.90s
2026-04-08 04:10:47.680731 | instance | Gathering Facts --------------------------------------------------------- 0.75s
2026-04-08 04:10:47.680933 | instance | Start loop devices ------------------------------------------------------ 0.74s
2026-04-08 04:10:47.681135 | instance | Generate endpoint skeleton for missing variables ------------------------ 0.73s
2026-04-08 04:10:47.681335 | instance | Gathering Facts --------------------------------------------------------- 0.70s
2026-04-08 04:10:47.681552 | instance | Gathering Facts --------------------------------------------------------- 0.70s
2026-04-08 04:10:47.681777 | instance | Configure short hostname ------------------------------------------------ 0.69s
2026-04-08 04:10:47.681983 | instance | Gathering Facts --------------------------------------------------------- 0.67s
2026-04-08 04:10:47.682188 | instance | Purge "snapd" package --------------------------------------------------- 0.64s
2026-04-08 04:10:47.682388 | instance | Start up service -------------------------------------------------------- 0.57s
2026-04-08 04:10:47.682611 | instance | Create devices for Ceph ------------------------------------------------- 0.55s
2026-04-08 04:10:47.682815 | instance | Write new Ceph control plane configuration file to disk ----------------- 0.53s
2026-04-08 04:10:47.683016 | instance | Set permissions on loopback devices ------------------------------------- 0.52s
2026-04-08 04:10:47.784555 | instance | INFO     [aio > prepare] Executed: Successful
2026-04-08 04:10:47.797343 | instance | INFO     [aio > converge] Executing
2026-04-08 04:10:50.742706 | instance |
2026-04-08 04:10:50.743166 | instance | PLAY [all] *********************************************************************
2026-04-08 04:10:50.743486 | instance |
2026-04-08 04:10:50.743815 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 04:10:50.744145 | instance | Wednesday 08 April 2026  04:10:50 +0000 (0:00:00.020)       0:00:00.021 *******
2026-04-08 04:10:52.041991 | instance | [WARNING]: Platform linux on host instance is using the discovered Python
2026-04-08 04:10:52.042404 | instance | interpreter at /usr/bin/python3.10, but future installation of another Python
2026-04-08 04:10:52.042774 | instance | interpreter could change the meaning of that path. See
2026-04-08 04:10:52.043104 | instance | https://docs.ansible.com/ansible-
2026-04-08 04:10:52.043432 | instance | core/2.17/reference_appendices/interpreter_discovery.html for more information.
2026-04-08 04:10:52.054758 | instance | ok: [instance]
2026-04-08 04:10:52.055100 | instance |
2026-04-08 04:10:52.055439 | instance | TASK [Fail if atmosphere_ceph_enabled is set] **********************************
2026-04-08 04:10:52.055795 | instance | Wednesday 08 April 2026  04:10:52 +0000 (0:00:01.312)       0:00:01.334 *******
2026-04-08 04:10:52.098202 | instance | skipping: [instance]
2026-04-08 04:10:52.098578 | instance |
2026-04-08 04:10:52.098924 | instance | TASK [Set a fact with the "atmosphere_images" for other plays] *****************
2026-04-08 04:10:52.099257 | instance | Wednesday 08 April 2026  04:10:52 +0000 (0:00:00.043)       0:00:01.377 *******
2026-04-08 04:10:52.288376 | instance | ok: [instance]
2026-04-08 04:10:52.288719 | instance |
2026-04-08 04:10:52.289058 | instance | PLAY [Deploy Ceph monitors & managers] *****************************************
2026-04-08 04:10:52.289399 | instance |
2026-04-08 04:10:52.289743 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 04:10:52.290078 | instance | Wednesday 08 April 2026  04:10:52 +0000 (0:00:00.190)       0:00:01.567 *******
2026-04-08 04:10:53.188047 | instance | ok: [instance]
2026-04-08 04:10:53.188461 | instance |
2026-04-08 04:10:53.188833 | instance | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-04-08 04:10:53.189215 | instance | Wednesday 08 April 2026  04:10:53 +0000 (0:00:00.898)       0:00:02.466 *******
2026-04-08 04:10:53.469516 | instance | ok: [instance]
2026-04-08 04:10:53.469713 | instance |
2026-04-08 04:10:53.469878 | instance | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-04-08 04:10:53.470037 | instance | Wednesday 08 April 2026  04:10:53 +0000 (0:00:00.279)       0:00:02.746 *******
2026-04-08 04:10:53.505104 | instance | skipping: [instance]
2026-04-08 04:10:53.505168 | instance |
2026-04-08 04:10:53.505175 | instance | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] ***
2026-04-08 04:10:53.505180 | instance | Wednesday 08 April 2026  04:10:53 +0000 (0:00:00.037)       0:00:02.784 *******
2026-04-08 04:10:53.796690 | instance | changed: [instance]
2026-04-08 04:10:53.796735 | instance |
2026-04-08 04:10:53.796742 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-04-08 04:10:53.796748 | instance | Wednesday 08 April 2026  04:10:53 +0000 (0:00:00.291)       0:00:03.075 *******
2026-04-08 04:10:53.870704 | instance | ok: [instance] => {
2026-04-08 04:10:53.870801 | instance |     "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.0/runc.amd64"
2026-04-08 04:10:53.870822 | instance | }
2026-04-08 04:10:53.870833 | instance |
2026-04-08 04:10:53.871106 | instance | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-04-08 04:10:53.871156 | instance | Wednesday 08 April 2026  04:10:53 +0000 (0:00:00.074)       0:00:03.149 *******
2026-04-08 04:10:54.689354 | instance | changed: [instance]
2026-04-08 04:10:54.689881 | instance |
2026-04-08 04:10:54.689946 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-04-08 04:10:54.689955 | instance | Wednesday 08 April 2026  04:10:54 +0000 (0:00:00.817)       0:00:03.967 *******
2026-04-08 04:10:54.746106 | instance | skipping: [instance]
2026-04-08 04:10:54.746212 | instance |
2026-04-08 04:10:54.746283 | instance | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-04-08 04:10:54.746447 | instance | Wednesday 08 April 2026  04:10:54 +0000 (0:00:00.057)       0:00:04.025 *******
2026-04-08 04:10:54.801084 | instance | skipping: [instance]
2026-04-08 04:10:54.801478 | instance |
2026-04-08 04:10:54.801702 | instance | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-04-08 04:10:54.801710 | instance | Wednesday 08 April 2026  04:10:54 +0000 (0:00:00.054)       0:00:04.080 *******
2026-04-08 04:10:55.014927 | instance | ok: [instance]
2026-04-08 04:10:55.014994 | instance |
2026-04-08 04:10:55.015107 | instance | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-04-08 04:10:55.015283 | instance | Wednesday 08 April 2026  04:10:55 +0000 (0:00:00.214)       0:00:04.294 *******
2026-04-08 04:10:56.514285 | instance | ok: [instance]
2026-04-08 04:10:56.514356 | instance |
2026-04-08 04:10:56.514478 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-04-08 04:10:56.514730 | instance | Wednesday 08 April 2026  04:10:56 +0000 (0:00:01.499)       0:00:05.793 *******
2026-04-08 04:10:56.584628 | instance | ok: [instance] => {
2026-04-08 04:10:56.584771 | instance |     "msg": "https://github.com/containerd/containerd/releases/download/v2.2.0/containerd-2.2.0-linux-amd64.tar.gz"
2026-04-08 04:10:56.585003 | instance | }
2026-04-08 04:10:56.585147 | instance |
2026-04-08 04:10:56.585300 | instance | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-04-08 04:10:56.585450 | instance | Wednesday 08 April 2026  04:10:56 +0000 (0:00:00.070)       0:00:05.863 *******
2026-04-08 04:10:57.299138 | instance | changed: [instance]
2026-04-08 04:10:57.299238 | instance |
2026-04-08 04:10:57.299255 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-04-08 04:10:57.299440 | instance | Wednesday 08 April 2026  04:10:57 +0000 (0:00:00.713)       0:00:06.577 *******
2026-04-08 04:11:00.219856 | instance | changed: [instance]
2026-04-08 04:11:00.219913 | instance |
2026-04-08 04:11:00.219919 | instance | TASK [vexxhost.containers.containerd : Install SELinux packages] ***************
2026-04-08 04:11:00.219925 | instance | Wednesday 08 April 2026  04:11:00 +0000 (0:00:02.920)       0:00:09.497 *******
2026-04-08 04:11:00.255691 | instance | skipping: [instance]
2026-04-08 04:11:00.256106 | instance |
2026-04-08 04:11:00.256338 | instance | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] ***
2026-04-08 04:11:00.256346 | instance | Wednesday 08 April 2026  04:11:00 +0000 (0:00:00.036)       0:00:09.534 *******
2026-04-08 04:11:00.297880 | instance | skipping: [instance]
2026-04-08 04:11:00.298262 | instance |
2026-04-08 04:11:00.298294 | instance | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ********
2026-04-08 04:11:00.298301 | instance | Wednesday 08 April 2026  04:11:00 +0000 (0:00:00.042)       0:00:09.577 *******
2026-04-08 04:11:00.339010 | instance | skipping: [instance]
2026-04-08 04:11:00.339113 | instance |
2026-04-08 04:11:00.339240 | instance | TASK [vexxhost.containers.containerd : Install AppArmor packages] **************
2026-04-08 04:11:00.339365 | instance | Wednesday 08 April 2026  04:11:00 +0000 (0:00:00.040)       0:00:09.618 *******
2026-04-08 04:11:05.566595 | instance | changed: [instance]
2026-04-08 04:11:05.566812 | instance |
2026-04-08 04:11:05.567098 | instance | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] ***
2026-04-08 04:11:05.567376 | instance | Wednesday 08 April 2026  04:11:05 +0000 (0:00:05.227)       0:00:14.845 *******
2026-04-08 04:11:06.243406 | instance | changed: [instance]
2026-04-08 04:11:06.243634 | instance |
2026-04-08 04:11:06.243932 | instance | TASK [vexxhost.containers.containerd : Create folders for configuration] *******
2026-04-08 04:11:06.244197 | instance | Wednesday 08 April 2026  04:11:06 +0000 (0:00:00.676)       0:00:15.522 *******
2026-04-08 04:11:07.176151 | instance | changed: [instance] => (item={'path': '/etc/containerd'})
2026-04-08 04:11:07.176436 | instance | changed: [instance] => (item={'path': '/var/lib/containerd', 'mode': '0o700'})
2026-04-08 04:11:07.176753 | instance | changed: [instance] => (item={'path': '/run/containerd', 'mode': '0o711'})
2026-04-08 04:11:07.177051 | instance | changed: [instance] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'})
2026-04-08 04:11:07.177311 | instance | changed: [instance] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'})
2026-04-08 04:11:07.177553 | instance |
2026-04-08 04:11:07.177817 | instance | TASK [vexxhost.containers.containerd : Create containerd config file] **********
2026-04-08 04:11:07.178090 | instance | Wednesday 08 April 2026  04:11:07 +0000 (0:00:00.932)       0:00:16.454 *******
2026-04-08 04:11:07.746413 | instance | changed: [instance]
2026-04-08 04:11:07.746683 | instance |
2026-04-08 04:11:07.746961 | instance | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ********
2026-04-08 04:11:07.747227 | instance | Wednesday 08 April 2026  04:11:07 +0000 (0:00:00.561)       0:00:17.016 *******
2026-04-08 04:11:07.747473 | instance |
2026-04-08 04:11:07.747717 | instance | RUNNING HANDLER [vexxhost.containers.containerd : Reload systemd] **************
2026-04-08 04:11:07.747977 | instance | Wednesday 08 April 2026  04:11:07 +0000 (0:00:00.008)       0:00:17.025 *******
2026-04-08 04:11:08.733639 | instance | ok: [instance]
2026-04-08 04:11:08.733714 | instance |
2026-04-08 04:11:08.733726 | instance | RUNNING HANDLER [vexxhost.containers.containerd : Restart containerd] **********
2026-04-08 04:11:08.733737 | instance | Wednesday 08 April 2026  04:11:08 +0000 (0:00:00.986)       0:00:18.011 *******
2026-04-08 04:11:09.183327 | instance | changed: [instance]
2026-04-08 04:11:09.183437 | instance |
2026-04-08 04:11:09.183502 | instance | TASK [vexxhost.containers.containerd : Enable and start service] ***************
2026-04-08 04:11:09.183649 | instance | Wednesday 08 April 2026  04:11:09 +0000 (0:00:00.450)       0:00:18.462 *******
2026-04-08 04:11:09.748619 | instance | changed: [instance]
2026-04-08 04:11:09.748710 | instance |
2026-04-08 04:11:09.748774 | instance | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-04-08 04:11:09.748945 | instance | Wednesday 08 April 2026  04:11:09 +0000 (0:00:00.565)       0:00:19.027 *******
2026-04-08 04:11:09.976552 | instance | ok: [instance]
2026-04-08 04:11:09.976641 | instance |
2026-04-08 04:11:09.976934 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-04-08 04:11:09.976991 | instance | Wednesday 08 April 2026  04:11:09 +0000 (0:00:00.227)       0:00:19.255 *******
2026-04-08 04:11:10.038767 | instance | ok: [instance] => {
2026-04-08 04:11:10.038870 | instance |     "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz"
2026-04-08 04:11:10.039352 | instance | }
2026-04-08 04:11:10.039377 | instance |
2026-04-08 04:11:10.039384 | instance | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-04-08 04:11:10.039390 | instance | Wednesday 08 April 2026  04:11:10 +0000 (0:00:00.062)       0:00:19.318 *******
2026-04-08 04:11:10.809212 | instance | changed: [instance]
2026-04-08 04:11:10.809279 | instance |
2026-04-08 04:11:10.809563 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-04-08 04:11:10.809623 | instance | Wednesday 08 April 2026  04:11:10 +0000 (0:00:00.770)       0:00:20.088 *******
2026-04-08 04:11:15.258355 | instance | changed: [instance]
2026-04-08 04:11:15.258433 | instance |
2026-04-08 04:11:15.258473 | instance | TASK [vexxhost.containers.docker : Install AppArmor packages] ******************
2026-04-08 04:11:15.258669 | instance | Wednesday 08 April 2026  04:11:15 +0000 (0:00:04.448)       0:00:24.536 *******
2026-04-08 04:11:16.487206 | instance | ok: [instance]
2026-04-08 04:11:16.487298 | instance |
2026-04-08 04:11:16.487314 | instance | TASK [vexxhost.containers.docker : Ensure group "docker" exists] ***************
2026-04-08 04:11:16.487482 | instance | Wednesday 08 April 2026  04:11:16 +0000 (0:00:01.229)       0:00:25.766 *******
2026-04-08 04:11:16.914276 | instance | changed: [instance]
2026-04-08 04:11:16.914356 | instance |
2026-04-08 04:11:16.914728 | instance | TASK [vexxhost.containers.docker : Create systemd service file for docker] *****
2026-04-08 04:11:16.915010 | instance | Wednesday 08 April 2026  04:11:16 +0000 (0:00:00.427)       0:00:26.193 *******
2026-04-08 04:11:17.381390 | instance | changed: [instance]
2026-04-08 04:11:17.381484 | instance |
2026-04-08 04:11:17.381545 | instance | TASK [vexxhost.containers.docker : Create folders for configuration] ***********
2026-04-08 04:11:17.381744 | instance | Wednesday 08 April 2026  04:11:17 +0000 (0:00:00.466)       0:00:26.660 *******
2026-04-08 04:11:17.943510 | instance | changed: [instance] => (item={'path': '/etc/docker'})
2026-04-08 04:11:17.943606 | instance | changed: [instance] => (item={'path': '/var/lib/docker', 'mode': '0o710'})
2026-04-08 04:11:17.943665 | instance | changed: [instance] => (item={'path': '/run/docker', 'mode': '0o711'})
2026-04-08 04:11:17.944086 | instance |
2026-04-08 04:11:17.944349 | instance | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ******
2026-04-08 04:11:17.944357 | instance | Wednesday 08 April 2026  04:11:17 +0000 (0:00:00.562)       0:00:27.222 *******
2026-04-08 04:11:18.406416 | instance | changed: [instance]
2026-04-08 04:11:18.406481 | instance |
2026-04-08 04:11:18.406763 | instance | TASK [vexxhost.containers.docker : Create docker daemon config file] ***********
2026-04-08 04:11:18.406817 | instance | Wednesday 08 April 2026  04:11:18 +0000 (0:00:00.463)       0:00:27.685 *******
2026-04-08 04:11:18.876943 | instance | changed: [instance]
2026-04-08 04:11:18.877023 | instance |
2026-04-08 04:11:18.877611 | instance | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************
2026-04-08 04:11:18.877659 | instance | Wednesday 08 April 2026  04:11:18 +0000 (0:00:00.461)       0:00:28.147 *******
2026-04-08 04:11:18.877666 | instance |
2026-04-08 04:11:18.877671 | instance | RUNNING HANDLER [vexxhost.containers.containerd : Reload systemd] **************
2026-04-08 04:11:18.877687 | instance | Wednesday 08 April 2026  04:11:18 +0000 (0:00:00.008)       0:00:28.155 *******
2026-04-08 04:11:19.557155 | instance | ok: [instance]
2026-04-08 04:11:19.557257 | instance |
2026-04-08 04:11:19.557329 | instance | RUNNING HANDLER [vexxhost.containers.docker : Restart docker] ******************
2026-04-08 04:11:19.557474 | instance | Wednesday 08 April 2026  04:11:19 +0000 (0:00:00.680)       0:00:28.836 *******
2026-04-08 04:11:20.993542 | instance | changed: [instance]
2026-04-08 04:11:20.993631 | instance |
2026-04-08 04:11:20.994068 | instance | TASK [vexxhost.containers.docker : Enable and start service] *******************
2026-04-08 04:11:20.994141 | instance | Wednesday 08 April 2026  04:11:20 +0000 (0:00:01.436)       0:00:30.272 *******
2026-04-08 04:11:21.557705 | instance | changed: [instance]
2026-04-08 04:11:21.557790 | instance |
2026-04-08 04:11:21.558038 | instance | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ******
2026-04-08 04:11:21.558094 | instance | Wednesday 08 April 2026  04:11:21 +0000 (0:00:00.564)       0:00:30.837 *******
2026-04-08 04:11:21.629088 | instance | ok: [instance] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/ubuntu-22.04.yml)
2026-04-08 04:11:21.629510 | instance |
2026-04-08 04:11:21.629542 | instance | TASK [vexxhost.ceph.cephadm : Install packages] ********************************
2026-04-08 04:11:21.629550 | instance | Wednesday 08 April 2026  04:11:21 +0000 (0:00:00.070)       0:00:30.907 *******
2026-04-08 04:11:26.423312 | instance | changed: [instance]
2026-04-08 04:11:26.423420 | instance |
2026-04-08 04:11:26.423488 | instance | TASK [vexxhost.ceph.cephadm : Ensure services are started] *********************
2026-04-08 04:11:26.423650 | instance | Wednesday 08 April 2026  04:11:26 +0000 (0:00:04.794)       0:00:35.702 *******
2026-04-08 04:11:27.122457 | instance | ok: [instance] => (item=chronyd)
2026-04-08 04:11:27.122621 | instance | ok: [instance] => (item=sshd)
2026-04-08 04:11:27.122634 | instance |
2026-04-08 04:11:27.122905 | instance | TASK [vexxhost.ceph.cephadm : Download "cephadm"] ******************************
2026-04-08 04:11:27.122969 | instance | Wednesday 08 April 2026  04:11:27 +0000 (0:00:00.698)       0:00:36.401 *******
2026-04-08 04:11:27.421646 | instance | changed: [instance]
2026-04-08 04:11:27.421741 | instance |
2026-04-08 04:11:27.421805 | instance | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ********************
2026-04-08 04:11:27.421957 | instance | Wednesday 08 April 2026  04:11:27 +0000 (0:00:00.299)       0:00:36.700 *******
2026-04-08 04:11:27.628404 | instance | ok: [instance]
2026-04-08 04:11:27.628494 | instance |
2026-04-08 04:11:27.628537 | instance | TASK [vexxhost.ceph.cephadm : Ensure "cephadm" user is present] ****************
2026-04-08 04:11:27.628705 | instance | Wednesday 08 April 2026  04:11:27 +0000 (0:00:00.206)       0:00:36.907 *******
2026-04-08 04:11:28.190926 | instance | changed: [instance]
2026-04-08 04:11:28.191407 | instance |
2026-04-08 04:11:28.191430 | instance | TASK [vexxhost.ceph.cephadm : Allow "cephadm" user to have passwordless sudo] ***
2026-04-08 04:11:28.191438 | instance | Wednesday 08 April 2026  04:11:28 +0000 (0:00:00.562)       0:00:37.469 *******
2026-04-08 04:11:28.626241 | instance | changed: [instance]
2026-04-08 04:11:28.626319 | instance |
2026-04-08 04:11:28.626579 | instance | TASK [vexxhost.ceph.mon : Get `cephadm ls` status] *****************************
2026-04-08 04:11:28.626639 | instance | Wednesday 08 April 2026  04:11:28 +0000 (0:00:00.435)       0:00:37.905 *******
2026-04-08 04:11:30.314449 | instance | ok: [instance]
2026-04-08 04:11:30.314533 | instance |
2026-04-08 04:11:30.314647 | instance | TASK [vexxhost.ceph.mon : Parse the `cephadm ls` output] ***********************
2026-04-08 04:11:30.314780 | instance | Wednesday 08 April 2026  04:11:30 +0000 (0:00:01.688)       0:00:39.593 *******
2026-04-08 04:11:30.368874 | instance | ok: [instance]
2026-04-08 04:11:30.369060 | instance |
2026-04-08 04:11:30.369332 | instance | TASK [vexxhost.ceph.mon : Assimilate existing configs in `ceph.conf`] **********
2026-04-08 04:11:30.369601 | instance | Wednesday 08 April 2026  04:11:30 +0000 (0:00:00.054)       0:00:39.647 *******
2026-04-08 04:11:30.408900 | instance | skipping: [instance]
2026-04-08 04:11:30.409021 | instance |
2026-04-08 04:11:30.409193 | instance | TASK [vexxhost.ceph.mon : Adopt monitor to cluster] ****************************
2026-04-08 04:11:30.409361 | instance | Wednesday 08 April 2026  04:11:30 +0000 (0:00:00.040)       0:00:39.688 *******
2026-04-08 04:11:30.453168 | instance | skipping: [instance]
2026-04-08 04:11:30.453394 | instance |
2026-04-08 04:11:30.453668 | instance | TASK [vexxhost.ceph.mon : Adopt manager to cluster] ****************************
2026-04-08 04:11:30.453967 | instance | Wednesday 08 April 2026  04:11:30 +0000 (0:00:00.044)       0:00:39.732 *******
2026-04-08 04:11:30.496411 | instance | skipping: [instance]
2026-04-08 04:11:30.496748 | instance |
2026-04-08 04:11:30.497081 | instance | TASK [vexxhost.ceph.mon : Enable "cephadm" mgr module] *************************
2026-04-08 04:11:30.497407 | instance | Wednesday 08 April 2026  04:11:30 +0000 (0:00:00.042)       0:00:39.775 *******
2026-04-08 04:11:30.542890 | instance | skipping: [instance]
2026-04-08 04:11:30.543184 | instance |
2026-04-08 04:11:30.543542 | instance | TASK [vexxhost.ceph.mon : Set orchestrator backend to "cephadm"] ***************
2026-04-08 04:11:30.543858 | instance | Wednesday 08 April 2026  04:11:30 +0000 (0:00:00.046)       0:00:39.821 *******
2026-04-08 04:11:30.586174 | instance | skipping: [instance]
2026-04-08 04:11:30.586474 | instance |
2026-04-08 04:11:30.586854 | instance | TASK [vexxhost.ceph.mon : Use `cephadm` user for cephadm] **********************
2026-04-08 04:11:30.587166 | instance | Wednesday 08 April 2026  04:11:30 +0000 (0:00:00.043)       0:00:39.865 *******
2026-04-08 04:11:30.628880 | instance | skipping: [instance]
2026-04-08 04:11:30.629156 | instance |
2026-04-08 04:11:30.629538 | instance | TASK [vexxhost.ceph.mon : Generate "cephadm" key] ******************************
2026-04-08 04:11:30.629850 | instance | Wednesday 08 April 2026  04:11:30 +0000 (0:00:00.042)       0:00:39.908 *******
2026-04-08 04:11:30.670374 | instance | skipping: [instance]
2026-04-08 04:11:30.670645 | instance |
2026-04-08 04:11:30.670919 | instance | TASK [vexxhost.ceph.mon : Set Ceph Monitor IP address] *************************
2026-04-08 04:11:30.671187 | instance | Wednesday 08 April 2026  04:11:30 +0000 (0:00:00.041)       0:00:39.949 *******
2026-04-08 04:11:30.799218 | instance | ok: [instance]
2026-04-08 04:11:30.799579 | instance |
2026-04-08 04:11:30.799902 | instance | TASK [vexxhost.ceph.mon : Check if any node is bootstrapped] *******************
2026-04-08 04:11:30.800172 | instance | Wednesday 08 April 2026  04:11:30 +0000 (0:00:00.128)       0:00:40.078 *******
2026-04-08 04:11:31.028573 | instance | ok: [instance] => (item=instance)
2026-04-08 04:11:31.028921 | instance |
2026-04-08 04:11:31.029273 | instance | TASK [vexxhost.ceph.mon : Select pre-existing bootstrap node if exists] ********
2026-04-08 04:11:31.029669 | instance | Wednesday 08 April 2026  04:11:31 +0000 (0:00:00.229)       0:00:40.307 *******
2026-04-08 04:11:31.086006 | instance | ok: [instance]
2026-04-08 04:11:31.086259 | instance |
2026-04-08 04:11:31.086616 | instance | TASK [vexxhost.ceph.mon : Bootstrap cluster] ***********************************
2026-04-08 04:11:31.086931 | instance | Wednesday 08 April 2026  04:11:31 +0000 (0:00:00.057)       0:00:40.364 *******
2026-04-08 04:11:31.172597 | instance | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/mon/tasks/bootstrap-ceph.yml for instance
2026-04-08 04:11:31.172876 | instance |
2026-04-08 04:11:31.173204 | instance | TASK [vexxhost.ceph.mon : Generate temporary file for "ceph.conf"] *************
2026-04-08 04:11:31.173526 | instance | Wednesday 08 April 2026  04:11:31 +0000 (0:00:00.086)       0:00:40.451 *******
2026-04-08 04:11:31.549205 | instance | changed: [instance]
2026-04-08 04:11:31.549386 | instance |
2026-04-08 04:11:31.549659 | instance | TASK [vexxhost.ceph.mon : Include extra configuration values] ******************
2026-04-08 04:11:31.549994 | instance | Wednesday 08 April 2026  04:11:31 +0000 (0:00:00.376)       0:00:40.828 *******
2026-04-08 04:11:32.293561 | instance | changed: [instance] => (item={'option': 'mon allow pool size one', 'section': 'global', 'value': True})
2026-04-08 04:11:32.293772 | instance | changed: [instance] => (item={'option': 'osd crush chooseleaf type', 'section': 'global', 'value': 0})
2026-04-08 04:11:32.294060 | instance | changed: [instance] => (item={'option': 'auth allow insecure global id reclaim', 'section': 'mon', 'value': False})
2026-04-08 04:11:32.294321 | instance |
2026-04-08 04:11:32.294663 | instance | TASK [vexxhost.ceph.mon : Run Bootstrap coomand] *******************************
2026-04-08 04:11:32.294991 | instance | Wednesday 08 April 2026  04:11:32 +0000 (0:00:00.744)       0:00:41.572 *******
2026-04-08 04:13:39.986630 | instance | ok: [instance]
2026-04-08 04:13:39.986878 | instance |
2026-04-08 04:13:39.987165 | instance | TASK [vexxhost.ceph.mon : Remove temporary file for "ceph.conf"] ***************
2026-04-08 04:13:39.987456 | instance | Wednesday 08 April 2026  04:13:39 +0000 (0:02:07.692)       0:02:49.264 *******
2026-04-08 04:13:40.197908 | instance | changed: [instance]
2026-04-08 04:13:40.198035 | instance |
2026-04-08 04:13:40.198209 | instance | TASK [vexxhost.ceph.mon : Set bootstrap node] **********************************
2026-04-08 04:13:40.198384 | instance | Wednesday 08 April 2026  04:13:40 +0000 (0:00:00.212)       0:02:49.477 *******
2026-04-08 04:13:40.248048 | instance | ok: [instance]
2026-04-08 04:13:40.249196 | instance |
2026-04-08 04:13:40.249216 | instance | TASK [Install Ceph host] *******************************************************
2026-04-08 04:13:40.249223 | instance | Wednesday 08 April 2026  04:13:40 +0000 (0:00:00.049)       0:02:49.526 *******
2026-04-08 04:13:40.335282 | instance | included: vexxhost.ceph.cephadm_host for instance
2026-04-08 04:13:40.335550 | instance |
2026-04-08 04:13:40.335822 | instance | TASK [vexxhost.ceph.cephadm_host : Get public SSH key for "cephadm" user] ******
2026-04-08 04:13:40.336126 | instance | Wednesday 08 April 2026  04:13:40 +0000 (0:00:00.087)       0:02:49.614 *******
2026-04-08 04:13:41.877277 | instance | ok: [instance]
2026-04-08 04:13:41.877499 | instance |
2026-04-08 04:13:41.877787 | instance | TASK [vexxhost.ceph.cephadm_host : Set fact with public SSH key for "cephadm" user] ***
2026-04-08 04:13:41.878057 | instance | Wednesday 08 April 2026  04:13:41 +0000 (0:00:01.541)       0:02:51.156 *******
2026-04-08 04:13:41.936529 | instance | ok: [instance] => (item=instance)
2026-04-08 04:13:41.936708 | instance |
2026-04-08 04:13:41.936968 | instance | TASK [vexxhost.ceph.cephadm_host : Set authorized key for "cephadm"] ***********
2026-04-08 04:13:41.937230 | instance | Wednesday 08 April 2026  04:13:41 +0000 (0:00:00.059)       0:02:51.215 *******
2026-04-08 04:13:42.370434 | instance | ok: [instance]
2026-04-08 04:13:42.370673 | instance |
2026-04-08 04:13:42.370944 | instance | TASK [vexxhost.ceph.cephadm_host : Add new host to Ceph] ***********************
2026-04-08 04:13:42.371236 | instance | Wednesday 08 April 2026  04:13:42 +0000 (0:00:00.433)       0:02:51.649 *******
2026-04-08 04:13:44.364154 | instance | ok: [instance]
2026-04-08 04:13:44.364403 | instance |
2026-04-08 04:13:44.364701 | instance | TASK [vexxhost.ceph.mon : Configure "mon" label for monitors] ******************
2026-04-08 04:13:44.364991 | instance | Wednesday 08 April 2026  04:13:44 +0000 (0:00:01.993)       0:02:53.643 *******
2026-04-08 04:13:45.953839 | instance | ok: [instance]
2026-04-08 04:13:45.953906 | instance |
2026-04-08 04:13:45.953918 | instance | TASK [vexxhost.ceph.mon : Validate monitor exist] ******************************
2026-04-08 04:13:45.953928 | instance | Wednesday 08 April 2026  04:13:45 +0000 (0:00:01.588)       0:02:55.232 *******
2026-04-08 04:13:57.167792 | instance | ok: [instance]
2026-04-08 04:13:57.167895 | instance |
2026-04-08 04:13:57.167909 | instance | TASK [Install Ceph host] *******************************************************
2026-04-08 04:13:57.167920 | instance | Wednesday 08 April 2026  04:13:57 +0000 (0:00:11.213)       0:03:06.445 *******
2026-04-08 04:13:57.244304 | instance | included: vexxhost.ceph.cephadm_host for instance
2026-04-08 04:13:57.244371 | instance |
2026-04-08 04:13:57.244639 | instance | TASK [vexxhost.ceph.cephadm_host : Get public SSH key for "cephadm" user] ******
2026-04-08 04:13:57.244677 | instance | Wednesday 08 April 2026  04:13:57 +0000 (0:00:00.078)       0:03:06.523 *******
2026-04-08 04:13:57.300772 | instance | skipping: [instance]
2026-04-08 04:13:57.301150 | instance |
2026-04-08 04:13:57.301192 | instance | TASK [vexxhost.ceph.cephadm_host : Set fact with public SSH key for "cephadm" user] ***
2026-04-08 04:13:57.301199 | instance | Wednesday 08 April 2026  04:13:57 +0000 (0:00:00.056)       0:03:06.580 *******
2026-04-08 04:13:57.354199 | instance | skipping: [instance] => (item=instance)
2026-04-08 04:13:57.354357 | instance | skipping: [instance]
2026-04-08 04:13:57.354533 | instance |
2026-04-08 04:13:57.354704 | instance | TASK [vexxhost.ceph.cephadm_host : Set authorized key for "cephadm"] ***********
2026-04-08 04:13:57.354867 | instance | Wednesday 08 April 2026  04:13:57 +0000 (0:00:00.053)       0:03:06.633 *******
2026-04-08 04:13:57.600590 | instance | ok: [instance]
2026-04-08 04:13:57.600806 | instance |
2026-04-08 04:13:57.601070 | instance | TASK [vexxhost.ceph.cephadm_host : Add new host to Ceph] ***********************
2026-04-08 04:13:57.601335 | instance | Wednesday 08 April 2026  04:13:57 +0000 (0:00:00.246)       0:03:06.879 *******
2026-04-08 04:13:59.601590 | instance | ok: [instance]
2026-04-08 04:13:59.601829 | instance |
2026-04-08 04:13:59.602111 | instance | TASK [vexxhost.ceph.mgr : Configure "mgr" label for managers] ******************
2026-04-08 04:13:59.602396 | instance | Wednesday 08 April 2026  04:13:59 +0000 (0:00:02.000)       0:03:08.880 *******
2026-04-08 04:14:01.145024 | instance | ok: [instance]
2026-04-08 04:14:01.145081 | instance |
2026-04-08 04:14:01.145093 | instance | TASK [vexxhost.ceph.mgr : Validate manager exist] ******************************
2026-04-08 04:14:01.145104 | instance | Wednesday 08 April 2026  04:14:01 +0000 (0:00:01.542)       0:03:10.423 *******
2026-04-08 04:14:02.684053 | instance | ok: [instance]
2026-04-08 04:14:02.684125 | instance |
2026-04-08 04:14:02.684408 | instance | TASK [vexxhost.ceph.mgr : Enable the Ceph Manager prometheus module] ***********
2026-04-08 04:14:02.684448 | instance | Wednesday 08 April 2026  04:14:02 +0000 (0:00:01.540)       0:03:11.963 *******
2026-04-08 04:14:04.356165 | instance | ok: [instance]
2026-04-08 04:14:04.356258 | instance |
2026-04-08 04:14:04.356334 | instance | PLAY [Deploy Ceph OSDs] ********************************************************
2026-04-08 04:14:04.356442 | instance |
2026-04-08 04:14:04.356565 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 04:14:04.356687 | instance | Wednesday 08 April 2026  04:14:04 +0000 (0:00:01.672)       0:03:13.635 *******
2026-04-08 04:14:05.238635 | instance | ok: [instance]
2026-04-08 04:14:05.238694 | instance |
2026-04-08 04:14:05.238833 | instance | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-04-08 04:14:05.238947 | instance | Wednesday 08 April 2026  04:14:05 +0000 (0:00:00.882)       0:03:14.517 *******
2026-04-08 04:14:05.438134 | instance | ok: [instance]
2026-04-08 04:14:05.438191 | instance |
2026-04-08 04:14:05.438310 | instance | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-04-08 04:14:05.438429 | instance | Wednesday 08 April 2026  04:14:05 +0000 (0:00:00.199)       0:03:14.717 *******
2026-04-08 04:14:05.475417 | instance | skipping: [instance]
2026-04-08 04:14:05.475456 | instance |
2026-04-08 04:14:05.475607 | instance | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] ***
2026-04-08 04:14:05.475725 | instance | Wednesday 08 April 2026  04:14:05 +0000 (0:00:00.037)       0:03:14.755 *******
2026-04-08 04:14:05.681279 | instance | ok: [instance]
2026-04-08 04:14:05.681322 | instance |
2026-04-08 04:14:05.681454 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-04-08 04:14:05.681628 | instance | Wednesday 08 April 2026  04:14:05 +0000 (0:00:00.205)       0:03:14.961 *******
2026-04-08 04:14:05.737795 | instance | ok: [instance] => {
2026-04-08 04:14:05.737932 | instance |     "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.0/runc.amd64"
2026-04-08 04:14:05.738139 | instance | }
2026-04-08 04:14:05.738228 | instance |
2026-04-08 04:14:05.738462 | instance | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-04-08 04:14:05.738598 | instance | Wednesday 08 April 2026  04:14:05 +0000 (0:00:00.056)       0:03:15.017 *******
2026-04-08 04:14:06.028202 | instance | ok: [instance]
2026-04-08 04:14:06.028269 | instance |
2026-04-08 04:14:06.028299 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-04-08 04:14:06.028423 | instance | Wednesday 08 April 2026  04:14:06 +0000 (0:00:00.290)       0:03:15.307 *******
2026-04-08 04:14:06.075713 | instance | skipping: [instance]
2026-04-08 04:14:06.075809 | instance |
2026-04-08 04:14:06.075917 | instance | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-04-08 04:14:06.076034 | instance | Wednesday 08 April 2026  04:14:06 +0000 (0:00:00.047)       0:03:15.355 *******
2026-04-08 04:14:06.125653 | instance | skipping: [instance]
2026-04-08 04:14:06.125716 | instance |
2026-04-08 04:14:06.125846 | instance | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-04-08 04:14:06.125964 | instance | Wednesday 08 April 2026  04:14:06 +0000 (0:00:00.049)       0:03:15.405 *******
2026-04-08 04:14:06.317341 | instance | ok: [instance]
2026-04-08 04:14:06.317385 | instance |
2026-04-08 04:14:06.317506 | instance | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-04-08 04:14:06.317617 | instance | Wednesday 08 April 2026  04:14:06 +0000 (0:00:00.191)       0:03:15.597 *******
2026-04-08 04:14:07.316657 | instance | ok: [instance]
2026-04-08 04:14:07.316729 | instance |
2026-04-08 04:14:07.316806 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-04-08 04:14:07.316976 | instance | Wednesday 08 April 2026  04:14:07 +0000 (0:00:00.998)       0:03:16.596 *******
2026-04-08 04:14:07.382762 | instance | ok: [instance] => {
2026-04-08 04:14:07.382916 | instance |     "msg": "https://github.com/containerd/containerd/releases/download/v2.2.0/containerd-2.2.0-linux-amd64.tar.gz"
2026-04-08 04:14:07.383083 | instance | }
2026-04-08 04:14:07.383242 | instance |
2026-04-08 04:14:07.383418 | instance | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-04-08 04:14:07.383583 | instance | Wednesday 08 April 2026  04:14:07 +0000 (0:00:00.066)       0:03:16.662 *******
2026-04-08 04:14:07.702336 | instance | ok: [instance]
2026-04-08 04:14:07.702397 | instance |
2026-04-08 04:14:07.702479 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-04-08 04:14:07.702624 | instance | Wednesday 08 April 2026  04:14:07 +0000 (0:00:00.319)       0:03:16.981 *******
2026-04-08 04:14:09.616305 | instance | ok: [instance]
2026-04-08 04:14:09.616377 | instance |
2026-04-08 04:14:09.616490 | instance | TASK [vexxhost.containers.containerd : Install SELinux packages] ***************
2026-04-08 04:14:09.616609 | instance | Wednesday 08 April 2026  04:14:09 +0000 (0:00:01.913)       0:03:18.895 *******
2026-04-08 04:14:09.648860 | instance | skipping: [instance]
2026-04-08 04:14:09.649036 | instance |
2026-04-08 04:14:09.649155 | instance | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] ***
2026-04-08 04:14:09.649278 | instance | Wednesday 08 April 2026  04:14:09 +0000 (0:00:00.032)       0:03:18.928 *******
2026-04-08 04:14:09.683397 | instance | skipping: [instance]
2026-04-08 04:14:09.683524 | instance |
2026-04-08 04:14:09.683691 | instance | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ********
2026-04-08 04:14:09.683876 | instance | Wednesday 08 April 2026  04:14:09 +0000 (0:00:00.034)       0:03:18.963 *******
2026-04-08 04:14:09.717711 | instance | skipping: [instance]
2026-04-08 04:14:09.717859 | instance |
2026-04-08 04:14:09.718026 | instance | TASK [vexxhost.containers.containerd : Install AppArmor packages] **************
2026-04-08 04:14:09.718192 | instance | Wednesday 08 April 2026  04:14:09 +0000 (0:00:00.034)       0:03:18.997 *******
2026-04-08 04:14:10.656431 | instance | ok: [instance]
2026-04-08 04:14:10.656523 | instance |
2026-04-08 04:14:10.656705 | instance | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] ***
2026-04-08 04:14:10.656822 | instance | Wednesday 08 April 2026  04:14:10 +0000 (0:00:00.938)       0:03:19.935 *******
2026-04-08 04:14:11.068947 | instance | ok: [instance]
2026-04-08 04:14:11.069032 | instance |
2026-04-08 04:14:11.069163 | instance | TASK [vexxhost.containers.containerd : Create folders for configuration] *******
2026-04-08 04:14:11.069290 | instance | Wednesday 08 April 2026  04:14:11 +0000 (0:00:00.412)       0:03:20.348 *******
2026-04-08 04:14:11.918160 | instance | ok: [instance] => (item={'path': '/etc/containerd'})
2026-04-08 04:14:11.918221 | instance | ok: [instance] => (item={'path': '/var/lib/containerd', 'mode': '0o700'})
2026-04-08 04:14:11.918356 | instance | ok: [instance] => (item={'path': '/run/containerd', 'mode': '0o711'})
2026-04-08 04:14:11.918537 | instance | ok: [instance] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'})
2026-04-08 04:14:11.918679 | instance | ok: [instance] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'})
2026-04-08 04:14:11.918797 | instance |
2026-04-08 04:14:11.918911 | instance | TASK [vexxhost.containers.containerd : Create containerd config file] **********
2026-04-08 04:14:11.919041 | instance | Wednesday 08 April 2026  04:14:11 +0000 (0:00:00.849)       0:03:21.197 *******
2026-04-08 04:14:12.393881 | instance | ok: [instance]
2026-04-08 04:14:12.393937 | instance |
2026-04-08 04:14:12.394059 | instance | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ********
2026-04-08 04:14:12.394180 | instance | Wednesday 08 April 2026  04:14:12 +0000 (0:00:00.468)       0:03:21.665 *******
2026-04-08 04:14:12.394297 | instance |
2026-04-08 04:14:12.394422 | instance | TASK [vexxhost.containers.containerd : Enable and start service] ***************
2026-04-08 04:14:12.394594 | instance | Wednesday 08 April 2026  04:14:12 +0000 (0:00:00.007)       0:03:21.673 *******
2026-04-08 04:14:12.740247 | instance | ok: [instance]
2026-04-08 04:14:12.740310 | instance |
2026-04-08 04:14:12.740441 | instance | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-04-08 04:14:12.740559 | instance | Wednesday 08 April 2026  04:14:12 +0000 (0:00:00.346)       0:03:22.019 *******
2026-04-08 04:14:12.941876 | instance | ok: [instance]
2026-04-08 04:14:12.941933 | instance |
2026-04-08 04:14:12.942052 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-04-08 04:14:12.942171 | instance | Wednesday 08 April 2026  04:14:12 +0000 (0:00:00.201)       0:03:22.221 *******
2026-04-08 04:14:12.992139 | instance | ok: [instance] => {
2026-04-08 04:14:12.992198 | instance |     "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz"
2026-04-08 04:14:12.992298 | instance | }
2026-04-08 04:14:12.992408 | instance |
2026-04-08 04:14:12.992589 | instance | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-04-08 04:14:12.992625 | instance | Wednesday 08 April 2026  04:14:12 +0000 (0:00:00.050)       0:03:22.271 *******
2026-04-08 04:14:13.340427 | instance | ok: [instance]
2026-04-08 04:14:13.340465 | instance |
2026-04-08 04:14:13.340655 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-04-08 04:14:13.340731 | instance | Wednesday 08 April 2026  04:14:13 +0000 (0:00:00.347)       0:03:22.619 *******
2026-04-08 04:14:16.361758 | instance | ok: [instance]
2026-04-08 04:14:16.361831 | instance |
2026-04-08 04:14:16.361949 | instance | TASK [vexxhost.containers.docker : Install AppArmor packages] ******************
2026-04-08 04:14:16.362066 | instance | Wednesday 08 April 2026  04:14:16 +0000 (0:00:03.020)       0:03:25.640 *******
2026-04-08 04:14:17.645968 | instance | ok: [instance]
2026-04-08 04:14:17.646112 | instance |
2026-04-08 04:14:17.646303 | instance | TASK [vexxhost.containers.docker : Ensure group "docker" exists] ***************
2026-04-08 04:14:17.646561 | instance | Wednesday 08 April 2026  04:14:17 +0000 (0:00:01.284)       0:03:26.925 *******
2026-04-08 04:14:17.856687 | instance | ok: [instance]
2026-04-08 04:14:17.856758 | instance |
2026-04-08 04:14:17.856884 | instance | TASK [vexxhost.containers.docker : Create systemd service file for docker] *****
2026-04-08 04:14:17.857008 | instance | Wednesday 08 April 2026  04:14:17 +0000 (0:00:00.210)       0:03:27.136 *******
2026-04-08 04:14:18.268150 | instance | ok: [instance]
2026-04-08 04:14:18.268213 | instance |
2026-04-08 04:14:18.268368 | instance | TASK [vexxhost.containers.docker : Create folders for configuration] ***********
2026-04-08 04:14:18.268452 | instance | Wednesday 08 April 2026  04:14:18 +0000 (0:00:00.411)       0:03:27.547 *******
2026-04-08 04:14:18.872327 | instance | ok: [instance] => (item={'path': '/etc/docker'})
2026-04-08 04:14:18.872398 | instance | ok: [instance] => (item={'path': '/var/lib/docker', 'mode': '0o710'})
2026-04-08 04:14:18.872508 | instance | ok: [instance] => (item={'path': '/run/docker', 'mode': '0o711'})
2026-04-08 04:14:18.872669 | instance |
2026-04-08 04:14:18.872740 | instance | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ******
2026-04-08 04:14:18.872876 | instance | Wednesday 08 April 2026  04:14:18 +0000 (0:00:00.604)       0:03:28.151 *******
2026-04-08 04:14:19.291835 | instance | ok: [instance]
2026-04-08 04:14:19.291942 | instance |
2026-04-08 04:14:19.292233 | instance | TASK [vexxhost.containers.docker : Create docker daemon config file] ***********
2026-04-08 04:14:19.292354 | instance | Wednesday 08 April 2026  04:14:19 +0000 (0:00:00.419)       0:03:28.571 *******
2026-04-08 04:14:19.704395 | instance | ok: [instance]
2026-04-08 04:14:19.705345 | instance |
2026-04-08 04:14:19.705405 | instance | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************
2026-04-08 04:14:19.705415 | instance | Wednesday 08 April 2026  04:14:19 +0000 (0:00:00.405)       0:03:28.976 *******
2026-04-08 04:14:19.705421 | instance |
2026-04-08 04:14:19.705427 | instance | TASK [vexxhost.containers.docker : Enable and start service] *******************
2026-04-08 04:14:19.705433 | instance | Wednesday 08 April 2026  04:14:19 +0000 (0:00:00.006)       0:03:28.983 *******
2026-04-08 04:14:20.073790 | instance | ok: [instance]
2026-04-08 04:14:20.073868 | instance |
2026-04-08 04:14:20.074128 | instance | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ******
2026-04-08 04:14:20.074172 | instance | Wednesday 08 April 2026  04:14:20 +0000 (0:00:00.369)       0:03:29.353 *******
2026-04-08 04:14:20.134142 | instance | ok: [instance] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/ubuntu-22.04.yml)
2026-04-08 04:14:20.134682 | instance |
2026-04-08 04:14:20.134811 | instance | TASK [vexxhost.ceph.cephadm : Install packages] ********************************
2026-04-08 04:14:20.134836 | instance | Wednesday 08 April 2026  04:14:20 +0000 (0:00:00.060)       0:03:29.413 *******
2026-04-08 04:14:21.089675 | instance | ok: [instance]
2026-04-08 04:14:21.089751 | instance |
2026-04-08 04:14:21.090125 | instance | TASK [vexxhost.ceph.cephadm : Ensure services are started] *********************
2026-04-08 04:14:21.090182 | instance | Wednesday 08 April 2026  04:14:21 +0000 (0:00:00.955)       0:03:30.368 *******
2026-04-08 04:14:21.708010 | instance | ok: [instance] => (item=chronyd)
2026-04-08 04:14:21.708139 | instance | ok: [instance] => (item=sshd)
2026-04-08 04:14:21.708155 | instance |
2026-04-08 04:14:21.708408 | instance | TASK [vexxhost.ceph.cephadm : Download "cephadm"] ******************************
2026-04-08 04:14:21.708451 | instance | Wednesday 08 April 2026  04:14:21 +0000 (0:00:00.618)       0:03:30.987 *******
2026-04-08 04:14:21.997614 | instance | ok: [instance]
2026-04-08 04:14:21.997693 | instance |
2026-04-08 04:14:21.997938 | instance | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ********************
2026-04-08 04:14:21.997980 | instance | Wednesday 08 April 2026  04:14:21 +0000 (0:00:00.289)       0:03:31.276 *******
2026-04-08 04:14:22.192196 | instance | ok: [instance]
2026-04-08 04:14:22.192278 | instance |
2026-04-08 04:14:22.192522 | instance | TASK [vexxhost.ceph.cephadm : Ensure "cephadm" user is present] ****************
2026-04-08 04:14:22.192563 | instance | Wednesday 08 April 2026  04:14:22 +0000 (0:00:00.194)       0:03:31.471 *******
2026-04-08 04:14:22.423312 | instance | ok: [instance]
2026-04-08 04:14:22.423659 | instance |
2026-04-08 04:14:22.423676 | instance | TASK [vexxhost.ceph.cephadm : Allow "cephadm" user to have passwordless sudo] ***
2026-04-08 04:14:22.423682 | instance | Wednesday 08 April 2026  04:14:22 +0000 (0:00:00.230)       0:03:31.702 *******
2026-04-08 04:14:22.616557 | instance | ok: [instance]
2026-04-08 04:14:22.616924 | instance |
2026-04-08 04:14:22.616967 | instance | TASK [vexxhost.ceph.osd : Get monitor status] **********************************
2026-04-08 04:14:22.616973 | instance | Wednesday 08 April 2026  04:14:22 +0000 (0:00:00.193)       0:03:31.895 *******
2026-04-08 04:14:22.830513 | instance | ok: [instance] => (item=instance)
2026-04-08 04:14:22.830636 | instance |
2026-04-08 04:14:22.830771 | instance | TASK [vexxhost.ceph.osd : Select admin host] ***********************************
2026-04-08 04:14:22.830909 | instance | Wednesday 08 April 2026  04:14:22 +0000 (0:00:00.213)       0:03:32.109 *******
2026-04-08 04:14:22.878705 | instance | ok: [instance]
2026-04-08 04:14:22.879083 | instance |
2026-04-08 04:14:22.879127 | instance | TASK [vexxhost.ceph.osd : Get `cephadm ls` status] *****************************
2026-04-08 04:14:22.879133 | instance | Wednesday 08 April 2026  04:14:22 +0000 (0:00:00.048)       0:03:32.158 *******
2026-04-08 04:14:27.710031 | instance | ok: [instance]
2026-04-08 04:14:27.710098 | instance |
2026-04-08 04:14:27.710350 | instance | TASK [vexxhost.ceph.osd : Parse the `cephadm ls` output] ***********************
2026-04-08 04:14:27.710390 | instance | Wednesday 08 April 2026  04:14:27 +0000 (0:00:04.831)       0:03:36.989 *******
2026-04-08 04:14:27.762343 | instance | ok: [instance]
2026-04-08 04:14:27.762557 | instance |
2026-04-08 04:14:27.762597 | instance | TASK [Install Ceph host] *******************************************************
2026-04-08 04:14:27.762735 | instance | Wednesday 08 April 2026  04:14:27 +0000 (0:00:00.052)       0:03:37.041 *******
2026-04-08 04:14:27.827867 | instance | included: vexxhost.ceph.cephadm_host for instance
2026-04-08 04:14:27.827906 | instance |
2026-04-08 04:14:27.828194 | instance | TASK [vexxhost.ceph.cephadm_host : Get public SSH key for "cephadm" user] ******
2026-04-08 04:14:27.828237 | instance | Wednesday 08 April 2026  04:14:27 +0000 (0:00:00.065)       0:03:37.107 *******
2026-04-08 04:14:27.871808 | instance | skipping: [instance]
2026-04-08 04:14:27.872170 | instance |
2026-04-08 04:14:27.872214 | instance | TASK [vexxhost.ceph.cephadm_host : Set fact with public SSH key for "cephadm" user] ***
2026-04-08 04:14:27.872222 | instance | Wednesday 08 April 2026  04:14:27 +0000 (0:00:00.043)       0:03:37.151 *******
2026-04-08 04:14:27.923101 | instance | skipping: [instance] => (item=instance)
2026-04-08 04:14:27.923277 | instance | skipping: [instance]
2026-04-08 04:14:27.923810 | instance |
2026-04-08 04:14:27.923836 | instance | TASK [vexxhost.ceph.cephadm_host : Set authorized key for "cephadm"] ***********
2026-04-08 04:14:27.923845 | instance | Wednesday 08 April 2026  04:14:27 +0000 (0:00:00.051)       0:03:37.202 *******
2026-04-08 04:14:28.173485 | instance | ok: [instance]
2026-04-08 04:14:28.173597 | instance |
2026-04-08 04:14:28.173664 | instance | TASK [vexxhost.ceph.cephadm_host : Add new host to Ceph] ***********************
2026-04-08 04:14:28.173803 | instance | Wednesday 08 April 2026  04:14:28 +0000 (0:00:00.250)       0:03:37.452 *******
2026-04-08 04:14:30.057108 | instance | ok: [instance]
2026-04-08 04:14:30.057265 | instance |
2026-04-08 04:14:30.057385 | instance | TASK [vexxhost.ceph.osd : Adopt OSDs to cluster] *******************************
2026-04-08 04:14:30.057504 | instance | Wednesday 08 April 2026  04:14:30 +0000 (0:00:01.883)       0:03:39.336 *******
2026-04-08 04:14:30.088981 | instance | skipping: [instance]
2026-04-08 04:14:30.089203 | instance |
2026-04-08 04:14:30.089477 | instance | TASK [vexxhost.ceph.osd : Wait until OSD added to cephadm] *********************
2026-04-08 04:14:30.089724 | instance | Wednesday 08 April 2026  04:14:30 +0000 (0:00:00.031)       0:03:39.368 *******
2026-04-08 04:14:30.118989 | instance | skipping: [instance]
2026-04-08 04:14:30.119199 | instance |
2026-04-08 04:14:30.119462 | instance | TASK [vexxhost.ceph.osd : Ensure all OSDs are non-legacy] **********************
2026-04-08 04:14:30.119723 | instance | Wednesday 08 April 2026  04:14:30 +0000 (0:00:00.030)       0:03:39.398 *******
2026-04-08 04:14:35.413803 | instance | ok: [instance]
2026-04-08 04:14:35.413999 | instance |
2026-04-08 04:14:35.414324 | instance | TASK [vexxhost.ceph.osd : Get `ceph-volume lvm list` status] *******************
2026-04-08 04:14:35.414639 | instance | Wednesday 08 April 2026  04:14:35 +0000 (0:00:05.294)       0:03:44.692 *******
2026-04-08 04:14:45.686669 | instance | ok: [instance]
2026-04-08 04:14:45.686738 | instance |
2026-04-08 04:14:45.686751 | instance | TASK [vexxhost.ceph.osd : Install OSDs] ****************************************
2026-04-08 04:14:45.686887 | instance | Wednesday 08 April 2026  04:14:45 +0000 (0:00:10.272)       0:03:54.965 *******
2026-04-08 04:14:53.440886 | instance | failed: [instance] (item=/dev/vdb) => {"ansible_loop_var": "item", "changed": false, "cmd": ["cephadm", "shell", "--fsid", "4837cbf8-4f90-4300-b3f6-726c9b9f89b4", "--config", "/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config", "--", "ceph", "orch", "daemon", "add", "osd", "instance:/dev/vdb"], "delta": "0:00:07.476199", "end": "2026-04-08 04:14:53.411543", "item": "/dev/vdb", "msg": "non-zero return code", "rc": 22, "start": "2026-04-08 04:14:45.935344", "stderr": "Error EINVAL: Traceback (most recent call last):\n  File \"/usr/share/ceph/mgr/mgr_module.py\", line 1834, in _handle_command\n    return self.handle_command(inbuf, cmd)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 183, in handle_command\n    return dispatch[cmd['prefix']].call(self, cmd, inbuf)\n  File \"/usr/share/ceph/mgr/mgr_module.py\", line 475, in call\n    return self.func(mgr, **kwargs)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 119, in <lambda>\n    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 108, in wrapper\n    return func(*args, **kwargs)\n  File \"/usr/share/ceph/mgr/orchestrator/module.py\", line 1306, in _daemon_add_osd\n    raise_if_exception(completion)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 240, in raise_if_exception\n    raise e\nRuntimeError: cephadm exited with an error code: 1, stderr:Inferring config /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config\nNon-zero exit code 1 from /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmppveqn32e:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmpeswjr_xx:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdb --yes --no-systemd\n/usr/bin/docker: stderr  stderr: lsblk: /dev/vdb: not a block device\n/usr/bin/docker: stderr Traceback (most recent call last):\n/usr/bin/docker: stderr   File \"/usr/sbin/ceph-volume\", line 33, in <module>\n/usr/bin/docker: stderr     sys.exit(load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')())\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 54, in __init__\n/usr/bin/docker: stderr     self.main(self.argv)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/decorators.py\", line 59, in newfunc\n/usr/bin/docker: stderr     return f(*a, **kw)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 166, in main\n/usr/bin/docker: stderr     terminal.dispatch(self.mapper, subcommand_args)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 194, in dispatch\n/usr/bin/docker: stderr     instance.main()\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/main.py\", line 46, in main\n/usr/bin/docker: stderr     terminal.dispatch(self.mapper, self.argv)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 192, in dispatch\n/usr/bin/docker: stderr     instance = mapper.get(arg)(argv[count:])\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/batch.py\", line 325, in __init__\n/usr/bin/docker: stderr     self.args = parser.parse_args(argv)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1825, in parse_args\n/usr/bin/docker: stderr     args, argv = self.parse_known_args(args, namespace)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1858, in parse_known_args\n/usr/bin/docker: stderr     namespace, args = self._parse_known_args(args, namespace)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2049, in _parse_known_args\n/usr/bin/docker: stderr     positionals_end_index = consume_positionals(start_index)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2026, in consume_positionals\n/usr/bin/docker: stderr     take_action(action, args)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1919, in take_action\n/usr/bin/docker: stderr     argument_values = self._get_values(action, argument_strings)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in _get_values\n/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in <listcomp>\n/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2483, in _get_value\n/usr/bin/docker: stderr     result = type_func(arg_string)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 124, in __call__\n/usr/bin/docker: stderr     super().get_device(dev_path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 32, in get_device\n/usr/bin/docker: stderr     self._device = Device(dev_path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 140, in __init__\n/usr/bin/docker: stderr     self._parse()\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 236, in _parse\n/usr/bin/docker: stderr     dev = disk.lsblk(self.path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 244, in lsblk\n/usr/bin/docker: stderr     result = lsblk_all(device=device,\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 338, in lsblk_all\n/usr/bin/docker: stderr     raise RuntimeError(f\"Error: {err}\")\n/usr/bin/docker: stderr RuntimeError: Error: ['lsblk: /dev/vdb: not a block device']\nTraceback (most recent call last):\n  File \"/usr/lib/python3.10/runpy.py\", line 196, in _run_module_as_main\n    return _run_code(code, main_globals, None,\n  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code\n    exec(code, run_globals)\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 11009, in <module>\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 10997, in main\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2593, in _infer_config\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2509, in _infer_fsid\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2621, in _infer_image\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2496, in _validate_fsid\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 7226, in command_ceph_volume\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2284, in call_throws\nRuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmppveqn32e:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmpeswjr_xx:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdb --yes --no-systemd", "stderr_lines": ["Error EINVAL: Traceback (most recent call last):", "  File \"/usr/share/ceph/mgr/mgr_module.py\", line 1834, in _handle_command", "    return self.handle_command(inbuf, cmd)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 183, in handle_command", "    return dispatch[cmd['prefix']].call(self, cmd, inbuf)", "  File \"/usr/share/ceph/mgr/mgr_module.py\", line 475, in call", "    return self.func(mgr, **kwargs)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 119, in <lambda>", "    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 108, in wrapper", "    return func(*args, **kwargs)", "  File \"/usr/share/ceph/mgr/orchestrator/module.py\", line 1306, in _daemon_add_osd", "    raise_if_exception(completion)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 240, in raise_if_exception", "    raise e", "RuntimeError: cephadm exited with an error code: 1, stderr:Inferring config /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config", "Non-zero exit code 1 from /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmppveqn32e:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmpeswjr_xx:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdb --yes --no-systemd", "/usr/bin/docker: stderr  stderr: lsblk: /dev/vdb: not a block device", "/usr/bin/docker: stderr Traceback (most recent call last):", "/usr/bin/docker: stderr   File \"/usr/sbin/ceph-volume\", line 33, in <module>", "/usr/bin/docker: stderr     sys.exit(load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')())", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 54, in __init__", "/usr/bin/docker: stderr     self.main(self.argv)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/decorators.py\", line 59, in newfunc", "/usr/bin/docker: stderr     return f(*a, **kw)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 166, in main", "/usr/bin/docker: stderr     terminal.dispatch(self.mapper, subcommand_args)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 194, in dispatch", "/usr/bin/docker: stderr     instance.main()", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/main.py\", line 46, in main", "/usr/bin/docker: stderr     terminal.dispatch(self.mapper, self.argv)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 192, in dispatch", "/usr/bin/docker: stderr     instance = mapper.get(arg)(argv[count:])", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/batch.py\", line 325, in __init__", "/usr/bin/docker: stderr     self.args = parser.parse_args(argv)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1825, in parse_args", "/usr/bin/docker: stderr     args, argv = self.parse_known_args(args, namespace)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1858, in parse_known_args", "/usr/bin/docker: stderr     namespace, args = self._parse_known_args(args, namespace)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2049, in _parse_known_args", "/usr/bin/docker: stderr     positionals_end_index = consume_positionals(start_index)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2026, in consume_positionals", "/usr/bin/docker: stderr     take_action(action, args)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1919, in take_action", "/usr/bin/docker: stderr     argument_values = self._get_values(action, argument_strings)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in _get_values", "/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in <listcomp>", "/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2483, in _get_value", "/usr/bin/docker: stderr     result = type_func(arg_string)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 124, in __call__", "/usr/bin/docker: stderr     super().get_device(dev_path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 32, in get_device", "/usr/bin/docker: stderr     self._device = Device(dev_path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 140, in __init__", "/usr/bin/docker: stderr     self._parse()", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 236, in _parse", "/usr/bin/docker: stderr     dev = disk.lsblk(self.path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 244, in lsblk", "/usr/bin/docker: stderr     result = lsblk_all(device=device,", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 338, in lsblk_all", "/usr/bin/docker: stderr     raise RuntimeError(f\"Error: {err}\")", "/usr/bin/docker: stderr RuntimeError: Error: ['lsblk: /dev/vdb: not a block device']", "Traceback (most recent call last):", "  File \"/usr/lib/python3.10/runpy.py\", line 196, in _run_module_as_main", "    return _run_code(code, main_globals, None,", "  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code", "    exec(code, run_globals)", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 11009, in <module>", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 10997, in main", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2593, in _infer_config", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2509, in _infer_fsid", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2621, in _infer_image", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2496, in _validate_fsid", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 7226, in command_ceph_volume", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2284, in call_throws", "RuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmppveqn32e:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmpeswjr_xx:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdb --yes --no-systemd"], "stdout": "", "stdout_lines": []}
2026-04-08 04:15:01.219689 | instance | failed: [instance] (item=/dev/vdc) => {"ansible_loop_var": "item", "changed": false, "cmd": ["cephadm", "shell", "--fsid", "4837cbf8-4f90-4300-b3f6-726c9b9f89b4", "--config", "/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config", "--", "ceph", "orch", "daemon", "add", "osd", "instance:/dev/vdc"], "delta": "0:00:07.595901", "end": "2026-04-08 04:15:01.198558", "item": "/dev/vdc", "msg": "non-zero return code", "rc": 22, "start": "2026-04-08 04:14:53.602657", "stderr": "Error EINVAL: Traceback (most recent call last):\n  File \"/usr/share/ceph/mgr/mgr_module.py\", line 1834, in _handle_command\n    return self.handle_command(inbuf, cmd)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 183, in handle_command\n    return dispatch[cmd['prefix']].call(self, cmd, inbuf)\n  File \"/usr/share/ceph/mgr/mgr_module.py\", line 475, in call\n    return self.func(mgr, **kwargs)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 119, in <lambda>\n    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 108, in wrapper\n    return func(*args, **kwargs)\n  File \"/usr/share/ceph/mgr/orchestrator/module.py\", line 1306, in _daemon_add_osd\n    raise_if_exception(completion)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 240, in raise_if_exception\n    raise e\nRuntimeError: cephadm exited with an error code: 1, stderr:Inferring config /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config\nNon-zero exit code 1 from /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpnz8borg8:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmpbi1545pb:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdc --yes --no-systemd\n/usr/bin/docker: stderr  stderr: lsblk: /dev/vdc: not a block device\n/usr/bin/docker: stderr Traceback (most recent call last):\n/usr/bin/docker: stderr   File \"/usr/sbin/ceph-volume\", line 33, in <module>\n/usr/bin/docker: stderr     sys.exit(load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')())\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 54, in __init__\n/usr/bin/docker: stderr     self.main(self.argv)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/decorators.py\", line 59, in newfunc\n/usr/bin/docker: stderr     return f(*a, **kw)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 166, in main\n/usr/bin/docker: stderr     terminal.dispatch(self.mapper, subcommand_args)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 194, in dispatch\n/usr/bin/docker: stderr     instance.main()\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/main.py\", line 46, in main\n/usr/bin/docker: stderr     terminal.dispatch(self.mapper, self.argv)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 192, in dispatch\n/usr/bin/docker: stderr     instance = mapper.get(arg)(argv[count:])\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/batch.py\", line 325, in __init__\n/usr/bin/docker: stderr     self.args = parser.parse_args(argv)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1825, in parse_args\n/usr/bin/docker: stderr     args, argv = self.parse_known_args(args, namespace)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1858, in parse_known_args\n/usr/bin/docker: stderr     namespace, args = self._parse_known_args(args, namespace)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2049, in _parse_known_args\n/usr/bin/docker: stderr     positionals_end_index = consume_positionals(start_index)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2026, in consume_positionals\n/usr/bin/docker: stderr     take_action(action, args)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1919, in take_action\n/usr/bin/docker: stderr     argument_values = self._get_values(action, argument_strings)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in _get_values\n/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in <listcomp>\n/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2483, in _get_value\n/usr/bin/docker: stderr     result = type_func(arg_string)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 124, in __call__\n/usr/bin/docker: stderr     super().get_device(dev_path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 32, in get_device\n/usr/bin/docker: stderr     self._device = Device(dev_path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 140, in __init__\n/usr/bin/docker: stderr     self._parse()\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 236, in _parse\n/usr/bin/docker: stderr     dev = disk.lsblk(self.path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 244, in lsblk\n/usr/bin/docker: stderr     result = lsblk_all(device=device,\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 338, in lsblk_all\n/usr/bin/docker: stderr     raise RuntimeError(f\"Error: {err}\")\n/usr/bin/docker: stderr RuntimeError: Error: ['lsblk: /dev/vdc: not a block device']\nTraceback (most recent call last):\n  File \"/usr/lib/python3.10/runpy.py\", line 196, in _run_module_as_main\n    return _run_code(code, main_globals, None,\n  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code\n    exec(code, run_globals)\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 11009, in <module>\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 10997, in main\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2593, in _infer_config\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2509, in _infer_fsid\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2621, in _infer_image\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2496, in _validate_fsid\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 7226, in command_ceph_volume\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2284, in call_throws\nRuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpnz8borg8:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmpbi1545pb:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdc --yes --no-systemd", "stderr_lines": ["Error EINVAL: Traceback (most recent call last):", "  File \"/usr/share/ceph/mgr/mgr_module.py\", line 1834, in _handle_command", "    return self.handle_command(inbuf, cmd)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 183, in handle_command", "    return dispatch[cmd['prefix']].call(self, cmd, inbuf)", "  File \"/usr/share/ceph/mgr/mgr_module.py\", line 475, in call", "    return self.func(mgr, **kwargs)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 119, in <lambda>", "    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 108, in wrapper", "    return func(*args, **kwargs)", "  File \"/usr/share/ceph/mgr/orchestrator/module.py\", line 1306, in _daemon_add_osd", "    raise_if_exception(completion)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 240, in raise_if_exception", "    raise e", "RuntimeError: cephadm exited with an error code: 1, stderr:Inferring config /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config", "Non-zero exit code 1 from /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpnz8borg8:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmpbi1545pb:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdc --yes --no-systemd", "/usr/bin/docker: stderr  stderr: lsblk: /dev/vdc: not a block device", "/usr/bin/docker: stderr Traceback (most recent call last):", "/usr/bin/docker: stderr   File \"/usr/sbin/ceph-volume\", line 33, in <module>", "/usr/bin/docker: stderr     sys.exit(load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')())", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 54, in __init__", "/usr/bin/docker: stderr     self.main(self.argv)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/decorators.py\", line 59, in newfunc", "/usr/bin/docker: stderr     return f(*a, **kw)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 166, in main", "/usr/bin/docker: stderr     terminal.dispatch(self.mapper, subcommand_args)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 194, in dispatch", "/usr/bin/docker: stderr     instance.main()", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/main.py\", line 46, in main", "/usr/bin/docker: stderr     terminal.dispatch(self.mapper, self.argv)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 192, in dispatch", "/usr/bin/docker: stderr     instance = mapper.get(arg)(argv[count:])", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/batch.py\", line 325, in __init__", "/usr/bin/docker: stderr     self.args = parser.parse_args(argv)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1825, in parse_args", "/usr/bin/docker: stderr     args, argv = self.parse_known_args(args, namespace)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1858, in parse_known_args", "/usr/bin/docker: stderr     namespace, args = self._parse_known_args(args, namespace)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2049, in _parse_known_args", "/usr/bin/docker: stderr     positionals_end_index = consume_positionals(start_index)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2026, in consume_positionals", "/usr/bin/docker: stderr     take_action(action, args)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1919, in take_action", "/usr/bin/docker: stderr     argument_values = self._get_values(action, argument_strings)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in _get_values", "/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in <listcomp>", "/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2483, in _get_value", "/usr/bin/docker: stderr     result = type_func(arg_string)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 124, in __call__", "/usr/bin/docker: stderr     super().get_device(dev_path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 32, in get_device", "/usr/bin/docker: stderr     self._device = Device(dev_path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 140, in __init__", "/usr/bin/docker: stderr     self._parse()", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 236, in _parse", "/usr/bin/docker: stderr     dev = disk.lsblk(self.path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 244, in lsblk", "/usr/bin/docker: stderr     result = lsblk_all(device=device,", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 338, in lsblk_all", "/usr/bin/docker: stderr     raise RuntimeError(f\"Error: {err}\")", "/usr/bin/docker: stderr RuntimeError: Error: ['lsblk: /dev/vdc: not a block device']", "Traceback (most recent call last):", "  File \"/usr/lib/python3.10/runpy.py\", line 196, in _run_module_as_main", "    return _run_code(code, main_globals, None,", "  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code", "    exec(code, run_globals)", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 11009, in <module>", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 10997, in main", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2593, in _infer_config", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2509, in _infer_fsid", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2621, in _infer_image", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2496, in _validate_fsid", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 7226, in command_ceph_volume", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2284, in call_throws", "RuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpnz8borg8:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmpbi1545pb:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdc --yes --no-systemd"], "stdout": "", "stdout_lines": []}
2026-04-08 04:15:08.910974 | instance | failed: [instance] (item=/dev/vdd) => {"ansible_loop_var": "item", "changed": false, "cmd": ["cephadm", "shell", "--fsid", "4837cbf8-4f90-4300-b3f6-726c9b9f89b4", "--config", "/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config", "--", "ceph", "orch", "daemon", "add", "osd", "instance:/dev/vdd"], "delta": "0:00:07.504049", "end": "2026-04-08 04:15:08.874137", "item": "/dev/vdd", "msg": "non-zero return code", "rc": 22, "start": "2026-04-08 04:15:01.370088", "stderr": "Error EINVAL: Traceback (most recent call last):\n  File \"/usr/share/ceph/mgr/mgr_module.py\", line 1834, in _handle_command\n    return self.handle_command(inbuf, cmd)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 183, in handle_command\n    return dispatch[cmd['prefix']].call(self, cmd, inbuf)\n  File \"/usr/share/ceph/mgr/mgr_module.py\", line 475, in call\n    return self.func(mgr, **kwargs)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 119, in <lambda>\n    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 108, in wrapper\n    return func(*args, **kwargs)\n  File \"/usr/share/ceph/mgr/orchestrator/module.py\", line 1306, in _daemon_add_osd\n    raise_if_exception(completion)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 240, in raise_if_exception\n    raise e\nRuntimeError: cephadm exited with an error code: 1, stderr:Inferring config /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config\nNon-zero exit code 1 from /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpo18ofy3r:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmp3bksfyv0:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdd --yes --no-systemd\n/usr/bin/docker: stderr  stderr: lsblk: /dev/vdd: not a block device\n/usr/bin/docker: stderr Traceback (most recent call last):\n/usr/bin/docker: stderr   File \"/usr/sbin/ceph-volume\", line 33, in <module>\n/usr/bin/docker: stderr     sys.exit(load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')())\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 54, in __init__\n/usr/bin/docker: stderr     self.main(self.argv)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/decorators.py\", line 59, in newfunc\n/usr/bin/docker: stderr     return f(*a, **kw)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 166, in main\n/usr/bin/docker: stderr     terminal.dispatch(self.mapper, subcommand_args)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 194, in dispatch\n/usr/bin/docker: stderr     instance.main()\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/main.py\", line 46, in main\n/usr/bin/docker: stderr     terminal.dispatch(self.mapper, self.argv)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 192, in dispatch\n/usr/bin/docker: stderr     instance = mapper.get(arg)(argv[count:])\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/batch.py\", line 325, in __init__\n/usr/bin/docker: stderr     self.args = parser.parse_args(argv)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1825, in parse_args\n/usr/bin/docker: stderr     args, argv = self.parse_known_args(args, namespace)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1858, in parse_known_args\n/usr/bin/docker: stderr     namespace, args = self._parse_known_args(args, namespace)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2049, in _parse_known_args\n/usr/bin/docker: stderr     positionals_end_index = consume_positionals(start_index)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2026, in consume_positionals\n/usr/bin/docker: stderr     take_action(action, args)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1919, in take_action\n/usr/bin/docker: stderr     argument_values = self._get_values(action, argument_strings)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in _get_values\n/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in <listcomp>\n/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2483, in _get_value\n/usr/bin/docker: stderr     result = type_func(arg_string)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 124, in __call__\n/usr/bin/docker: stderr     super().get_device(dev_path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 32, in get_device\n/usr/bin/docker: stderr     self._device = Device(dev_path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 140, in __init__\n/usr/bin/docker: stderr     self._parse()\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 236, in _parse\n/usr/bin/docker: stderr     dev = disk.lsblk(self.path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 244, in lsblk\n/usr/bin/docker: stderr     result = lsblk_all(device=device,\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 338, in lsblk_all\n/usr/bin/docker: stderr     raise RuntimeError(f\"Error: {err}\")\n/usr/bin/docker: stderr RuntimeError: Error: ['lsblk: /dev/vdd: not a block device']\nTraceback (most recent call last):\n  File \"/usr/lib/python3.10/runpy.py\", line 196, in _run_module_as_main\n    return _run_code(code, main_globals, None,\n  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code\n    exec(code, run_globals)\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 11009, in <module>\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 10997, in main\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2593, in _infer_config\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2509, in _infer_fsid\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2621, in _infer_image\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2496, in _validate_fsid\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 7226, in command_ceph_volume\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2284, in call_throws\nRuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpo18ofy3r:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmp3bksfyv0:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdd --yes --no-systemd", "stderr_lines": ["Error EINVAL: Traceback (most recent call last):", "  File \"/usr/share/ceph/mgr/mgr_module.py\", line 1834, in _handle_command", "    return self.handle_command(inbuf, cmd)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 183, in handle_command", "    return dispatch[cmd['prefix']].call(self, cmd, inbuf)", "  File \"/usr/share/ceph/mgr/mgr_module.py\", line 475, in call", "    return self.func(mgr, **kwargs)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 119, in <lambda>", "    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 108, in wrapper", "    return func(*args, **kwargs)", "  File \"/usr/share/ceph/mgr/orchestrator/module.py\", line 1306, in _daemon_add_osd", "    raise_if_exception(completion)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 240, in raise_if_exception", "    raise e", "RuntimeError: cephadm exited with an error code: 1, stderr:Inferring config /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config", "Non-zero exit code 1 from /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpo18ofy3r:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmp3bksfyv0:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdd --yes --no-systemd", "/usr/bin/docker: stderr  stderr: lsblk: /dev/vdd: not a block device", "/usr/bin/docker: stderr Traceback (most recent call last):", "/usr/bin/docker: stderr   File \"/usr/sbin/ceph-volume\", line 33, in <module>", "/usr/bin/docker: stderr     sys.exit(load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')())", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 54, in __init__", "/usr/bin/docker: stderr     self.main(self.argv)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/decorators.py\", line 59, in newfunc", "/usr/bin/docker: stderr     return f(*a, **kw)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 166, in main", "/usr/bin/docker: stderr     terminal.dispatch(self.mapper, subcommand_args)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 194, in dispatch", "/usr/bin/docker: stderr     instance.main()", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/main.py\", line 46, in main", "/usr/bin/docker: stderr     terminal.dispatch(self.mapper, self.argv)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 192, in dispatch", "/usr/bin/docker: stderr     instance = mapper.get(arg)(argv[count:])", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/batch.py\", line 325, in __init__", "/usr/bin/docker: stderr     self.args = parser.parse_args(argv)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1825, in parse_args", "/usr/bin/docker: stderr     args, argv = self.parse_known_args(args, namespace)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1858, in parse_known_args", "/usr/bin/docker: stderr     namespace, args = self._parse_known_args(args, namespace)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2049, in _parse_known_args", "/usr/bin/docker: stderr     positionals_end_index = consume_positionals(start_index)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2026, in consume_positionals", "/usr/bin/docker: stderr     take_action(action, args)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1919, in take_action", "/usr/bin/docker: stderr     argument_values = self._get_values(action, argument_strings)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in _get_values", "/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in <listcomp>", "/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2483, in _get_value", "/usr/bin/docker: stderr     result = type_func(arg_string)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 124, in __call__", "/usr/bin/docker: stderr     super().get_device(dev_path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 32, in get_device", "/usr/bin/docker: stderr     self._device = Device(dev_path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 140, in __init__", "/usr/bin/docker: stderr     self._parse()", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 236, in _parse", "/usr/bin/docker: stderr     dev = disk.lsblk(self.path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 244, in lsblk", "/usr/bin/docker: stderr     result = lsblk_all(device=device,", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 338, in lsblk_all", "/usr/bin/docker: stderr     raise RuntimeError(f\"Error: {err}\")", "/usr/bin/docker: stderr RuntimeError: Error: ['lsblk: /dev/vdd: not a block device']", "Traceback (most recent call last):", "  File \"/usr/lib/python3.10/runpy.py\", line 196, in _run_module_as_main", "    return _run_code(code, main_globals, None,", "  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code", "    exec(code, run_globals)", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 11009, in <module>", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 10997, in main", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2593, in _infer_config", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2509, in _infer_fsid", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2621, in _infer_image", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2496, in _validate_fsid", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 7226, in command_ceph_volume", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2284, in call_throws", "RuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpo18ofy3r:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmp3bksfyv0:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdd --yes --no-systemd"], "stdout": "", "stdout_lines": []}
2026-04-08 04:15:08.917731 | instance |
2026-04-08 04:15:08.918530 | instance | PLAY RECAP *********************************************************************
2026-04-08 04:15:08.918547 | instance | instance                   : ok=105  changed=26   unreachable=0    failed=1    skipped=26   rescued=0    ignored=0
2026-04-08 04:15:08.918557 | instance |
2026-04-08 04:15:08.918566 | instance | Wednesday 08 April 2026  04:15:08 +0000 (0:00:23.231)       0:04:18.196 *******
2026-04-08 04:15:08.922151 | instance | ===============================================================================
2026-04-08 04:15:08.922229 | instance | vexxhost.ceph.mon : Run Bootstrap coomand ----------------------------- 127.69s
2026-04-08 04:15:08.922236 | instance | vexxhost.ceph.osd : Install OSDs --------------------------------------- 23.23s
2026-04-08 04:15:08.922243 | instance | vexxhost.ceph.mon : Validate monitor exist ----------------------------- 11.21s
2026-04-08 04:15:08.922249 | instance | vexxhost.ceph.osd : Get `ceph-volume lvm list` status ------------------ 10.27s
2026-04-08 04:15:08.922254 | instance | vexxhost.ceph.osd : Ensure all OSDs are non-legacy ---------------------- 5.29s
2026-04-08 04:15:08.922261 | instance | vexxhost.containers.containerd : Install AppArmor packages -------------- 5.23s
2026-04-08 04:15:08.922267 | instance | vexxhost.ceph.osd : Get `cephadm ls` status ----------------------------- 4.83s
2026-04-08 04:15:08.922273 | instance | vexxhost.ceph.cephadm : Install packages -------------------------------- 4.79s
2026-04-08 04:15:08.922278 | instance | vexxhost.containers.download_artifact : Extract archive ----------------- 4.45s
2026-04-08 04:15:08.922284 | instance | vexxhost.containers.download_artifact : Extract archive ----------------- 3.02s
2026-04-08 04:15:08.922290 | instance | vexxhost.containers.download_artifact : Extract archive ----------------- 2.92s
2026-04-08 04:15:08.922295 | instance | vexxhost.ceph.cephadm_host : Add new host to Ceph ----------------------- 2.00s
2026-04-08 04:15:08.922301 | instance | vexxhost.ceph.cephadm_host : Add new host to Ceph ----------------------- 1.99s
2026-04-08 04:15:08.922306 | instance | vexxhost.containers.download_artifact : Extract archive ----------------- 1.91s
2026-04-08 04:15:08.922312 | instance | vexxhost.ceph.cephadm_host : Add new host to Ceph ----------------------- 1.88s
2026-04-08 04:15:08.922318 | instance | vexxhost.ceph.mon : Get `cephadm ls` status ----------------------------- 1.69s
2026-04-08 04:15:08.922323 | instance | vexxhost.ceph.mgr : Enable the Ceph Manager prometheus module ----------- 1.67s
2026-04-08 04:15:08.922329 | instance | vexxhost.containers.containerd : Reload systemd ------------------------- 1.67s
2026-04-08 04:15:08.922335 | instance | vexxhost.ceph.mon : Configure "mon" label for monitors ------------------ 1.59s
2026-04-08 04:15:08.922340 | instance | vexxhost.ceph.mgr : Configure "mgr" label for managers ------------------ 1.54s
2026-04-08 04:15:09.113372 | instance | CRITICAL Ansible return code was 2, command was: ansible-playbook --inventory /home/zuul/.ansible/tmp/molecule.v9Wo.aio/inventory --skip-tags molecule-notest,notest --inventory=/home/zuul/src/github.com/vexxhost/atmosphere/inventory.yaml /home/zuul/src/github.com/vexxhost/atmosphere/molecule/aio/converge.yml
2026-04-08 04:15:09.113625 | instance | ERROR    [aio > converge] Executed: Failed
2026-04-08 04:15:09.113763 | instance | ERROR    Ansible return code was 2, command was: ansible-playbook --inventory /home/zuul/.ansible/tmp/molecule.v9Wo.aio/inventory --skip-tags molecule-notest,notest --inventory=/home/zuul/src/github.com/vexxhost/atmosphere/inventory.yaml /home/zuul/src/github.com/vexxhost/atmosphere/molecule/aio/converge.yml
2026-04-08 04:15:09.291322 | instance | ERROR
2026-04-08 04:15:09.291556 | instance | {
2026-04-08 04:15:09.291585 | instance |   "delta": "0:06:37.222834",
2026-04-08 04:15:09.291605 | instance |   "end": "2026-04-08 04:15:09.230823",
2026-04-08 04:15:09.291624 | instance |   "msg": "non-zero return code",
2026-04-08 04:15:09.291641 | instance |   "rc": 2,
2026-04-08 04:15:09.291662 | instance |   "start": "2026-04-08 04:08:32.007989"
2026-04-08 04:15:09.291680 | instance | }
failure
2026-04-08 04:15:09.306824 | 
2026-04-08 04:15:09.306896 | PLAY RECAP
2026-04-08 04:15:09.306965 | instance | ok: 2 changed: 2 unreachable: 0 failed: 1 skipped: 0 rescued: 0 ignored: 0
2026-04-08 04:15:09.306989 | 
2026-04-08 04:15:09.440620 | RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/run.yaml@main]
2026-04-08 04:15:09.450638 | POST-RUN START: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/post.yml@main]
2026-04-08 04:15:10.067844 | 
2026-04-08 04:15:10.068235 | PLAY [all]
2026-04-08 04:15:10.083274 | 
2026-04-08 04:15:10.083363 | TASK [gather-host-logs : creating directory for system status]
2026-04-08 04:15:10.424022 | instance | changed
2026-04-08 04:15:10.432011 | 
2026-04-08 04:15:10.432131 | TASK [gather-host-logs : Get logs for each host]
2026-04-08 04:15:10.784340 | instance | + systemd-cgls --full --all --no-pager
2026-04-08 04:15:10.799165 | instance | + ip addr
2026-04-08 04:15:10.801954 | instance | + ip route
2026-04-08 04:15:10.804750 | instance | + lsblk
2026-04-08 04:15:10.808011 | instance | + mount
2026-04-08 04:15:10.810650 | instance | + docker images
2026-04-08 04:15:10.829455 | instance | + brctl show
2026-04-08 04:15:10.829991 | instance | /bin/bash: line 8: brctl: command not found
2026-04-08 04:15:10.830252 | instance | + ps aux --sort=-%mem
2026-04-08 04:15:10.851614 | instance | + dpkg -l
2026-04-08 04:15:10.862859 | instance | + CONTAINERS=($(docker ps -a --format '{{ .Names }}' --filter label=zuul))
2026-04-08 04:15:10.863411 | instance | ++ docker ps -a --format '{{ .Names }}' --filter label=zuul
2026-04-08 04:15:10.884268 | instance | + '[' '!' -z '' ']'
2026-04-08 04:15:10.981661 | instance | ok: Runtime: 0:00:00.104979
2026-04-08 04:15:10.990837 | 
2026-04-08 04:15:10.990940 | TASK [gather-host-logs : Downloads logs to executor]
2026-04-08 04:15:11.636007 | instance | changed:
2026-04-08 04:15:11.636195 | instance | created directory /var/lib/zuul/builds/99af325baabc4fd49cb71d4bc4f2ea3f/work/logs/instance
2026-04-08 04:15:11.636222 | instance | cd+++++++++ system/
2026-04-08 04:15:11.636243 | instance | >f+++++++++ system/brctl-show.txt
2026-04-08 04:15:11.636263 | instance | >f+++++++++ system/docker-images.txt
2026-04-08 04:15:11.636282 | instance | >f+++++++++ system/ip-addr.txt
2026-04-08 04:15:11.636304 | instance | >f+++++++++ system/ip-route.txt
2026-04-08 04:15:11.636326 | instance | >f+++++++++ system/lsblk.txt
2026-04-08 04:15:11.636345 | instance | >f+++++++++ system/mount.txt
2026-04-08 04:15:11.636364 | instance | >f+++++++++ system/packages.txt
2026-04-08 04:15:11.636382 | instance | >f+++++++++ system/ps.txt
2026-04-08 04:15:11.636402 | instance | >f+++++++++ system/systemd-cgls.txt
2026-04-08 04:15:11.645646 | 
2026-04-08 04:15:11.645714 | LOOP [helm-release-status : creating directory for helm release status]
2026-04-08 04:15:11.838254 | instance | changed: "values"
2026-04-08 04:15:12.002029 | instance | changed: "releases"
2026-04-08 04:15:12.023274 | 
2026-04-08 04:15:12.023472 | TASK [helm-release-status : Gather get release status for helm charts]
2026-04-08 04:15:12.227861 | instance | /bin/bash: line 3: kubectl: command not found
2026-04-08 04:15:12.559040 | instance | ok: Runtime: 0:00:00.006306
2026-04-08 04:15:12.566874 | 
2026-04-08 04:15:12.566970 | TASK [helm-release-status : Downloads logs to executor]
2026-04-08 04:15:13.106357 | instance | changed:
2026-04-08 04:15:13.106625 | instance | cd+++++++++ helm/
2026-04-08 04:15:13.106686 | instance | cd+++++++++ helm/releases/
2026-04-08 04:15:13.106735 | instance | cd+++++++++ helm/values/
2026-04-08 04:15:13.117250 | 
2026-04-08 04:15:13.117325 | TASK [describe-kubernetes-objects : creating directory for cluster scoped objects]
2026-04-08 04:15:13.312918 | instance | changed
2026-04-08 04:15:13.353757 | 
2026-04-08 04:15:13.353917 | TASK [describe-kubernetes-objects : Gathering descriptions for cluster scoped objects]
2026-04-08 04:15:13.562569 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
2026-04-08 04:15:13.563302 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
2026-04-08 04:15:13.568724 | instance | environment: line 1: kubectl: command not found
2026-04-08 04:15:13.569532 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value
2026-04-08 04:15:13.571067 | instance | environment: line 1: kubectl: command not found
2026-04-08 04:15:13.571954 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value
2026-04-08 04:15:13.574590 | instance | environment: line 1: kubectl: command not found
2026-04-08 04:15:13.576292 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value
2026-04-08 04:15:13.577707 | instance | environment: line 1: kubectl: command not found
2026-04-08 04:15:13.579428 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value
2026-04-08 04:15:13.580094 | instance | environment: line 1: kubectl: command not found
2026-04-08 04:15:13.580964 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value
2026-04-08 04:15:13.889841 | instance | ok: Runtime: 0:00:00.024434
2026-04-08 04:15:13.896699 | 
2026-04-08 04:15:13.896764 | TASK [describe-kubernetes-objects : creating directory for namespace scoped objects]
2026-04-08 04:15:14.104128 | instance | changed
2026-04-08 04:15:14.109223 | 
2026-04-08 04:15:14.109295 | TASK [describe-kubernetes-objects : Gathering descriptions for namespace scoped objects]
2026-04-08 04:15:14.314604 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
2026-04-08 04:15:14.314751 | instance | environment: line 5: kubectl: command not found
2026-04-08 04:15:14.315570 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
2026-04-08 04:15:14.316046 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
2026-04-08 04:15:14.646714 | instance | ok: Runtime: 0:00:00.009826
2026-04-08 04:15:14.652725 | 
2026-04-08 04:15:14.652787 | TASK [describe-kubernetes-objects : Downloads logs to executor]
2026-04-08 04:15:15.134084 | instance | changed:
2026-04-08 04:15:15.134319 | instance | cd+++++++++ objects/
2026-04-08 04:15:15.134353 | instance | cd+++++++++ objects/cluster/
2026-04-08 04:15:15.134376 | instance | cd+++++++++ objects/namespaced/
2026-04-08 04:15:15.143944 | 
2026-04-08 04:15:15.144008 | TASK [gather-pod-logs : creating directory for pod logs]
2026-04-08 04:15:15.329704 | instance | changed
2026-04-08 04:15:15.336375 | 
2026-04-08 04:15:15.336441 | TASK [gather-pod-logs : creating directory for failed pod logs]
2026-04-08 04:15:15.545175 | instance | changed
2026-04-08 04:15:15.550268 | 
2026-04-08 04:15:15.550335 | TASK [gather-pod-logs : retrieve all kubernetes logs, current and previous (if they exist)]
2026-04-08 04:15:15.750810 | instance | environment: line 3: kubectl: command not found
2026-04-08 04:15:16.083178 | instance | ok: Runtime: 0:00:00.008958
2026-04-08 04:15:16.088309 | 
2026-04-08 04:15:16.088374 | TASK [gather-pod-logs : Downloads pod logs to executor]
2026-04-08 04:15:16.575535 | instance | changed:
2026-04-08 04:15:16.575705 | instance | cd+++++++++ pod-logs/
2026-04-08 04:15:16.575742 | instance | cd+++++++++ pod-logs/failed-pods/
2026-04-08 04:15:16.585140 | 
2026-04-08 04:15:16.585201 | TASK [gather-prom-metrics : creating directory for helm release descriptions]
2026-04-08 04:15:16.778851 | instance | changed
2026-04-08 04:15:16.785174 | 
2026-04-08 04:15:16.785238 | TASK [gather-prom-metrics : Get metrics from exporter services in all namespaces]
2026-04-08 04:15:17.003637 | instance | /bin/bash: line 2: kubectl: command not found
2026-04-08 04:15:17.317573 | instance | ok: Runtime: 0:00:00.038238
2026-04-08 04:15:17.322458 | 
2026-04-08 04:15:17.322525 | TASK [gather-prom-metrics : Get ceph metrics from ceph-mgr]
2026-04-08 04:15:17.519677 | instance | /bin/bash: line 2: kubectl: command not found
2026-04-08 04:15:17.548439 | instance | ceph-mgr endpoints:
2026-04-08 04:15:17.857395 | instance | ok: Runtime: 0:00:00.032960
2026-04-08 04:15:17.864456 | 
2026-04-08 04:15:17.864526 | TASK [gather-prom-metrics : Get metrics from fluentd pods]
2026-04-08 04:15:18.079818 | instance | /bin/bash: line 4: kubectl: command not found
2026-04-08 04:15:18.401379 | instance | ok: Runtime: 0:00:00.030524
2026-04-08 04:15:18.407791 | 
2026-04-08 04:15:18.407858 | TASK [gather-prom-metrics : Downloads logs to executor]
2026-04-08 04:15:18.900536 | instance | changed: cd+++++++++ prometheus/
2026-04-08 04:15:18.910160 | 
2026-04-08 04:15:18.910229 | TASK [gather-selenium-data : creating directory for helm release descriptions]
2026-04-08 04:15:19.148846 | instance | changed
2026-04-08 04:15:19.154072 | 
2026-04-08 04:15:19.154135 | TASK [gather-selenium-data : Get selenium data]
2026-04-08 04:15:19.359360 | instance | + cp '/tmp/artifacts/*' /tmp/logs/selenium/.
2026-04-08 04:15:19.360816 | instance | cp: cannot stat '/tmp/artifacts/*': No such file or directory
2026-04-08 04:15:19.691656 | instance | ERROR
2026-04-08 04:15:19.692032 | instance | {
2026-04-08 04:15:19.692093 | instance |   "delta": "0:00:00.006427",
2026-04-08 04:15:19.692124 | instance |   "end": "2026-04-08 04:15:19.361164",
2026-04-08 04:15:19.692154 | instance |   "msg": "non-zero return code",
2026-04-08 04:15:19.692181 | instance |   "rc": 1,
2026-04-08 04:15:19.692206 | instance |   "start": "2026-04-08 04:15:19.354737"
2026-04-08 04:15:19.692231 | instance | }
2026-04-08 04:15:19.692263 | instance | ERROR: Ignoring Errors
2026-04-08 04:15:19.699229 | 
2026-04-08 04:15:19.699337 | TASK [gather-selenium-data : Downloads logs to executor]
2026-04-08 04:15:20.182655 | instance | changed: cd+++++++++ selenium/
2026-04-08 04:15:20.190717 | 
2026-04-08 04:15:20.190806 | PLAY RECAP
2026-04-08 04:15:20.190884 | instance | ok: 23 changed: 23 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 1
2026-04-08 04:15:20.190923 | 
2026-04-08 04:15:20.325371 | POST-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/post.yml@main]
2026-04-08 04:15:20.335417 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main]
2026-04-08 04:15:20.952251 | 
2026-04-08 04:15:20.952382 | PLAY [all]
2026-04-08 04:15:20.963443 | 
2026-04-08 04:15:20.963522 | TASK [fetch-output : Set log path for multiple nodes]
2026-04-08 04:15:21.009433 | instance | skipping: Conditional result was False
2026-04-08 04:15:21.020528 | 
2026-04-08 04:15:21.020635 | TASK [fetch-output : Set log path for single node]
2026-04-08 04:15:21.066105 | instance | ok
2026-04-08 04:15:21.073102 | 
2026-04-08 04:15:21.073204 | LOOP [fetch-output : Ensure local output dirs]
2026-04-08 04:15:21.506472 | instance -> localhost | ok: "/var/lib/zuul/builds/99af325baabc4fd49cb71d4bc4f2ea3f/work/logs"
2026-04-08 04:15:21.757164 | instance -> localhost | changed: "/var/lib/zuul/builds/99af325baabc4fd49cb71d4bc4f2ea3f/work/artifacts"
2026-04-08 04:15:21.967183 | instance -> localhost | changed: "/var/lib/zuul/builds/99af325baabc4fd49cb71d4bc4f2ea3f/work/docs"
2026-04-08 04:15:21.982471 | 
2026-04-08 04:15:21.982631 | LOOP [fetch-output : Collect logs, artifacts and docs]
2026-04-08 04:15:22.622941 | instance | changed: .d..t...... ./
2026-04-08 04:15:22.623309 | instance | changed: All items complete
2026-04-08 04:15:22.623416 | 
2026-04-08 04:15:23.083624 | instance | changed: .d..t...... ./
2026-04-08 04:15:23.538709 | instance | changed: .d..t...... ./
2026-04-08 04:15:23.565905 | 
2026-04-08 04:15:23.566572 | LOOP [merge-output-to-logs : Move artifacts and docs to logs dir]
2026-04-08 04:15:24.000110 | instance -> localhost | ok: Item: artifacts Runtime: 0:00:00.007768
2026-04-08 04:15:24.224877 | instance -> localhost | ok: Item: docs Runtime: 0:00:00.007381
2026-04-08 04:15:24.238678 | 
2026-04-08 04:15:24.238827 | PLAY [all]
2026-04-08 04:15:24.246236 | 
2026-04-08 04:15:24.246300 | TASK [remove-build-sshkey : Remove the build SSH key from all nodes]
2026-04-08 04:15:24.666200 | instance | changed
2026-04-08 04:15:24.673747 | 
2026-04-08 04:15:24.673819 | PLAY RECAP
2026-04-08 04:15:24.673882 | instance | ok: 5 changed: 4 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0
2026-04-08 04:15:24.673911 | 
2026-04-08 04:15:24.816226 | POST-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main]
2026-04-08 04:15:24.828169 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post-logs.yaml@main]
2026-04-08 04:15:25.417296 | 
2026-04-08 04:15:25.417466 | PLAY [localhost]
2026-04-08 04:15:25.427611 | 
2026-04-08 04:15:25.427687 | TASK [Generate Zuul manifest]
2026-04-08 04:15:25.450223 | localhost | ok
2026-04-08 04:15:25.468267 | 
2026-04-08 04:15:25.468345 | TASK [generate-zuul-manifest : Generate Zuul manifest]
2026-04-08 04:15:25.834414 | localhost | changed
2026-04-08 04:15:25.845015 | 
2026-04-08 04:15:25.845099 | TASK [generate-zuul-manifest : Return Zuul manifest URL to Zuul]
2026-04-08 04:15:25.878307 | localhost | ok
2026-04-08 04:15:25.888091 | 
2026-04-08 04:15:25.888155 | TASK [Upload logs]
2026-04-08 04:15:25.910927 | localhost | ok
2026-04-08 04:15:26.014989 | 
2026-04-08 04:15:26.015138 | TASK [Set zuul-log-path fact]
2026-04-08 04:15:26.039003 | localhost | ok
2026-04-08 04:15:26.052698 | 
2026-04-08 04:15:26.052770 | TASK [set-zuul-log-path-fact : Set log path for a build]
2026-04-08 04:15:26.085440 | localhost | ok
2026-04-08 04:15:26.096728 | 
2026-04-08 04:15:26.096819 | TASK [upload-logs : Create log directories]
2026-04-08 04:15:26.454183 | localhost | changed
2026-04-08 04:15:26.460632 | 
2026-04-08 04:15:26.460728 | TASK [upload-logs : Ensure logs are readable before uploading]
2026-04-08 04:15:26.818925 | localhost -> localhost | ok: Runtime: 0:00:00.006560
2026-04-08 04:15:26.826451 | 
2026-04-08 04:15:26.826550 | TASK [upload-logs : Upload logs to log server]
2026-04-08 04:15:27.321312 | localhost | Output suppressed because no_log was given
2026-04-08 04:15:27.324975 | 
2026-04-08 04:15:27.325042 | LOOP [upload-logs : Compress console log and json output]
2026-04-08 04:15:27.368708 | localhost | skipping: Conditional result was False
2026-04-08 04:15:27.376172 | localhost | skipping: Conditional result was False
2026-04-08 04:15:27.389113 | 
2026-04-08 04:15:27.389189 | LOOP [upload-logs : Upload compressed console log and json output]
2026-04-08 04:15:27.429099 | localhost | skipping: Conditional result was False
2026-04-08 04:15:27.429295 | 
2026-04-08 04:15:27.433864 | localhost | skipping: Conditional result was False
2026-04-08 04:15:27.444242 | 
2026-04-08 04:15:27.444384 | LOOP [upload-logs : Upload console log and json output]