2026-04-08 04:07:54.385675 | Job console starting
2026-04-08 04:07:54.404660 | Updating git repos
2026-04-08 04:07:54.590713 | Cloning repos into workspace
2026-04-08 04:07:55.218090 | Restoring repo states
2026-04-08 04:07:55.238442 | Merging changes
2026-04-08 04:07:56.441364 | Checking out repos
2026-04-08 04:07:56.610127 | Preparing playbooks
2026-04-08 04:08:00.798434 | Running Ansible setup
2026-04-08 04:08:04.176528 | PRE-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main]
2026-04-08 04:08:04.815874 | 
2026-04-08 04:08:04.816411 | PLAY [localhost]
2026-04-08 04:08:04.823516 | 
2026-04-08 04:08:04.823586 | TASK [Gathering Facts]
2026-04-08 04:08:05.636110 | localhost | ok
2026-04-08 04:08:05.646919 | 
2026-04-08 04:08:05.647021 | TASK [Setup log path fact]
2026-04-08 04:08:05.666127 | localhost | ok
2026-04-08 04:08:05.679196 | 
2026-04-08 04:08:05.679285 | TASK [set-zuul-log-path-fact : Set log path for a build]
2026-04-08 04:08:05.707940 | localhost | ok
2026-04-08 04:08:05.718636 | 
2026-04-08 04:08:05.718707 | TASK [emit-job-header : Print job information]
2026-04-08 04:08:05.758191 | # Job Information
2026-04-08 04:08:05.758371 | Ansible Version: 2.16.16
2026-04-08 04:08:05.758415 | Job: atmosphere-molecule-aio-openvswitch
2026-04-08 04:08:05.758446 | Pipeline: check
2026-04-08 04:08:05.758474 | Executor: 0a8996d2b663
2026-04-08 04:08:05.758502 | Triggered by: https://github.com/vexxhost/atmosphere/pull/3809
2026-04-08 04:08:05.758533 | Event ID: 6dbdd510-3300-11f1-95e4-ee2cd8b60bfe
2026-04-08 04:08:05.762448 | 
2026-04-08 04:08:05.762544 | LOOP [emit-job-header : Print node information]
2026-04-08 04:08:05.855467 | localhost | ok:
2026-04-08 04:08:05.855790 | localhost | # Node Information
2026-04-08 04:08:05.855851 | localhost | Inventory Hostname: instance
2026-04-08 04:08:05.855898 | localhost | Hostname: np0000164193
2026-04-08 04:08:05.855942 | localhost | Username: zuul
2026-04-08 04:08:05.855991 | localhost | Distro: Ubuntu 22.04
2026-04-08 04:08:05.856035 | localhost | Provider: yul1
2026-04-08 04:08:05.856138 | localhost | Region: ca-ymq-1
2026-04-08 04:08:05.856197 | localhost | Label: ubuntu-jammy-16
2026-04-08 04:08:05.856240 | localhost | Product Name: OpenStack Nova
2026-04-08 04:08:05.856281 | localhost | Interface IP: 162.253.55.221
2026-04-08 04:08:05.869516 | 
2026-04-08 04:08:05.869662 | TASK [log-inventory : Ensure Zuul Ansible directory exists]
2026-04-08 04:08:06.238535 | localhost -> localhost | changed
2026-04-08 04:08:06.247872 | 
2026-04-08 04:08:06.247971 | TASK [log-inventory : Copy ansible inventory to logs dir]
2026-04-08 04:08:07.063069 | localhost -> localhost | changed
2026-04-08 04:08:07.068977 | 
2026-04-08 04:08:07.069033 | PLAY [all]
2026-04-08 04:08:07.076901 | 
2026-04-08 04:08:07.076983 | TASK [add-build-sshkey : Check to see if ssh key was already created for this build]
2026-04-08 04:08:07.338974 | instance -> localhost | ok
2026-04-08 04:08:07.345239 | 
2026-04-08 04:08:07.345303 | TASK [add-build-sshkey : Create a new key in workspace based on build UUID]
2026-04-08 04:08:07.380423 | instance | ok
2026-04-08 04:08:07.395870 | instance | included: /var/lib/zuul/builds/aed307efefdd4e9b8baf020d172c9ed9/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml
2026-04-08 04:08:07.401490 | 
2026-04-08 04:08:07.401550 | TASK [add-build-sshkey : Create Temp SSH key]
2026-04-08 04:08:08.091547 | instance -> localhost | Generating public/private rsa key pair.
2026-04-08 04:08:08.091733 | instance -> localhost | Your identification has been saved in /var/lib/zuul/builds/aed307efefdd4e9b8baf020d172c9ed9/work/aed307efefdd4e9b8baf020d172c9ed9_id_rsa
2026-04-08 04:08:08.091772 | instance -> localhost | Your public key has been saved in /var/lib/zuul/builds/aed307efefdd4e9b8baf020d172c9ed9/work/aed307efefdd4e9b8baf020d172c9ed9_id_rsa.pub
2026-04-08 04:08:08.091803 | instance -> localhost | The key fingerprint is:
2026-04-08 04:08:08.091831 | instance -> localhost | SHA256:R52H7VKluyCs/TowgXcXl6pnJVGaYKGuJNlHx+1KXvk zuul-build-sshkey
2026-04-08 04:08:08.091874 | instance -> localhost | The key's randomart image is:
2026-04-08 04:08:08.091902 | instance -> localhost | +---[RSA 3072]----+
2026-04-08 04:08:08.091935 | instance -> localhost | |          +. ....|
2026-04-08 04:08:08.091964 | instance -> localhost | |         + +o*oo |
2026-04-08 04:08:08.091991 | instance -> localhost | |       .o + B==  |
2026-04-08 04:08:08.092018 | instance -> localhost | |     o.oo+..+=.. |
2026-04-08 04:08:08.092044 | instance -> localhost | |    o o.So=o*oo  |
2026-04-08 04:08:08.092071 | instance -> localhost | |     o oo*.+o+ . |
2026-04-08 04:08:08.092097 | instance -> localhost | |      . .o+o  E  |
2026-04-08 04:08:08.092123 | instance -> localhost | |          ..     |
2026-04-08 04:08:08.092153 | instance -> localhost | |          .o.    |
2026-04-08 04:08:08.092180 | instance -> localhost | +----[SHA256]-----+
2026-04-08 04:08:08.092242 | instance -> localhost | ok: Runtime: 0:00:00.282664
2026-04-08 04:08:08.098960 | 
2026-04-08 04:08:08.099049 | TASK [add-build-sshkey : Remote setup ssh keys (linux)]
2026-04-08 04:08:08.148296 | instance | ok
2026-04-08 04:08:08.157184 | instance | included: /var/lib/zuul/builds/aed307efefdd4e9b8baf020d172c9ed9/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml
2026-04-08 04:08:08.164944 | 
2026-04-08 04:08:08.165026 | TASK [add-build-sshkey : Remove previously added zuul-build-sshkey]
2026-04-08 04:08:08.189364 | instance | skipping: Conditional result was False
2026-04-08 04:08:08.198712 | 
2026-04-08 04:08:08.198897 | TASK [add-build-sshkey : Enable access via build key on all nodes]
2026-04-08 04:08:08.613878 | instance | changed
2026-04-08 04:08:08.618784 | 
2026-04-08 04:08:08.618853 | TASK [add-build-sshkey : Make sure user has a .ssh]
2026-04-08 04:08:08.803415 | instance | ok
2026-04-08 04:08:08.808360 | 
2026-04-08 04:08:08.808441 | TASK [add-build-sshkey : Install build private key as SSH key on all nodes]
2026-04-08 04:08:09.262650 | instance | changed
2026-04-08 04:08:09.267470 | 
2026-04-08 04:08:09.267534 | TASK [add-build-sshkey : Install build public key as SSH key on all nodes]
2026-04-08 04:08:09.718499 | instance | changed
2026-04-08 04:08:09.728893 | 
2026-04-08 04:08:09.729006 | TASK [add-build-sshkey : Remote setup ssh keys (windows)]
2026-04-08 04:08:09.755311 | instance | skipping: Conditional result was False
2026-04-08 04:08:09.774435 | 
2026-04-08 04:08:09.774502 | TASK [remove-zuul-sshkey : Remove master key from local agent]
2026-04-08 04:08:10.133064 | instance -> localhost | changed
2026-04-08 04:08:10.179600 | 
2026-04-08 04:08:10.179718 | TASK [add-build-sshkey : Add back temp key]
2026-04-08 04:08:10.456365 | instance -> localhost | Identity added: /var/lib/zuul/builds/aed307efefdd4e9b8baf020d172c9ed9/work/aed307efefdd4e9b8baf020d172c9ed9_id_rsa (zuul-build-sshkey)
2026-04-08 04:08:10.456716 | instance -> localhost | ok: Runtime: 0:00:00.014089
2026-04-08 04:08:10.464430 | 
2026-04-08 04:08:10.464563 | TASK [add-build-sshkey : Verify we can still SSH to all nodes]
2026-04-08 04:08:10.726956 | instance | ok
2026-04-08 04:08:10.732871 | 
2026-04-08 04:08:10.733002 | TASK [add-build-sshkey : Verify we can still SSH to all nodes (windows)]
2026-04-08 04:08:10.757638 | instance | skipping: Conditional result was False
2026-04-08 04:08:10.774303 | 
2026-04-08 04:08:10.774435 | TASK [prepare-workspace : Start zuul_console daemon.]
2026-04-08 04:08:11.042289 | instance | ok
2026-04-08 04:08:11.047745 | 
2026-04-08 04:08:11.047813 | TASK [prepare-workspace : Synchronize src repos to workspace directory.]
2026-04-08 04:08:12.642562 | instance | Output suppressed because no_log was given
2026-04-08 04:08:12.653667 | 
2026-04-08 04:08:12.653753 | LOOP [ensure-output-dirs : Empty Zuul Output directories by removing them]
2026-04-08 04:08:12.861182 | instance | ok: "logs"
2026-04-08 04:08:12.861493 | instance | ok: All items complete
2026-04-08 04:08:12.861532 | 
2026-04-08 04:08:13.014531 | instance | ok: "artifacts"
2026-04-08 04:08:13.177273 | instance | ok: "docs"
2026-04-08 04:08:13.185881 | 
2026-04-08 04:08:13.186059 | LOOP [ensure-output-dirs : Ensure Zuul Output directories exist]
2026-04-08 04:08:13.380438 | instance | changed: "logs"
2026-04-08 04:08:13.526193 | instance | changed: "artifacts"
2026-04-08 04:08:13.685977 | instance | changed: "docs"
2026-04-08 04:08:13.705675 | 
2026-04-08 04:08:13.705827 | PLAY RECAP
2026-04-08 04:08:13.705882 | instance | ok: 15 changed: 8 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0
2026-04-08 04:08:13.705911 | localhost | ok: 6 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0
2026-04-08 04:08:13.705932 | 
2026-04-08 04:08:13.908043 | PRE-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main]
2026-04-08 04:08:13.920367 | PRE-RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main]
2026-04-08 04:08:14.517750 | 
2026-04-08 04:08:14.517878 | PLAY [all]
2026-04-08 04:08:14.529933 | 
2026-04-08 04:08:14.530007 | TASK [setup-uv : Extract archive]
2026-04-08 04:08:16.619796 | instance | changed
2026-04-08 04:08:16.628160 | 
2026-04-08 04:08:16.628270 | TASK [setup-uv : Print version]
2026-04-08 04:08:16.412958 | instance | uv 0.8.13
2026-04-08 04:08:17.166512 | instance | ok: Runtime: 0:00:00.012399
2026-04-08 04:08:17.172439 | 
2026-04-08 04:08:17.172500 | PLAY RECAP
2026-04-08 04:08:17.172543 | instance | ok: 2 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0
2026-04-08 04:08:17.172567 | 
2026-04-08 04:08:17.290021 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main]
2026-04-08 04:08:17.302675 | PRE-RUN START: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/pre.yml@main]
2026-04-08 04:08:17.960995 | 
2026-04-08 04:08:17.961128 | PLAY [all]
2026-04-08 04:08:17.973350 | 
2026-04-08 04:08:17.973435 | TASK [Install "jq" for log collection]
2026-04-08 04:08:42.956869 | instance | changed
2026-04-08 04:08:42.958637 | 
2026-04-08 04:08:42.958693 | PLAY RECAP
2026-04-08 04:08:42.958741 | instance | ok: 1 changed: 1 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0
2026-04-08 04:08:42.958790 | 
2026-04-08 04:08:43.055186 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/pre.yml@main]
2026-04-08 04:08:43.067346 | RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/run.yaml@main]
2026-04-08 04:08:43.647170 | 
2026-04-08 04:08:43.647299 | PLAY [all]
2026-04-08 04:08:43.658805 | 
2026-04-08 04:08:43.658878 | TASK [Copy inventory file for Zuul]
2026-04-08 04:08:44.441246 | instance | changed
2026-04-08 04:08:44.449333 | 
2026-04-08 04:08:44.449496 | TASK [Switch "ansible_host" to private IP]
2026-04-08 04:08:44.770221 | instance | changed: 1 replacements made
2026-04-08 04:08:44.776878 | 
2026-04-08 04:08:44.777005 | TASK [Run Molecule scenario]
2026-04-08 04:08:45.142480 | instance | Using CPython 3.10.12 interpreter at: /usr/bin/python3
2026-04-08 04:08:45.142589 | instance | Creating virtual environment at: .venv
2026-04-08 04:08:45.172560 | instance |    Building atmosphere @ file:///home/zuul/src/github.com/vexxhost/atmosphere
2026-04-08 04:08:45.209357 | instance | Downloading kubernetes (1.9MiB)
2026-04-08 04:08:45.209822 | instance | Downloading openstacksdk (1.7MiB)
2026-04-08 04:08:45.210067 | instance | Downloading ansible-core (2.1MiB)
2026-04-08 04:08:45.210588 | instance | Downloading pygments (1.2MiB)
2026-04-08 04:08:45.210839 | instance | Downloading pydantic-core (2.0MiB)
2026-04-08 04:08:45.211511 | instance | Downloading netaddr (2.2MiB)
2026-04-08 04:08:45.211822 | instance | Downloading setuptools (1.1MiB)
2026-04-08 04:08:45.212339 | instance | Downloading rjsonnet (1.2MiB)
2026-04-08 04:08:45.213119 | instance | Downloading cryptography (4.2MiB)
2026-04-08 04:08:45.508635 | instance |    Building pyperclip==1.9.0
2026-04-08 04:08:45.532840 | instance |  Downloading rjsonnet
2026-04-08 04:08:45.616387 | instance |  Downloading pydantic-core
2026-04-08 04:08:45.661481 | instance |  Downloading netaddr
2026-04-08 04:08:45.672011 | instance |  Downloading pygments
2026-04-08 04:08:45.684439 | instance |  Downloading cryptography
2026-04-08 04:08:45.718170 | instance |  Downloading setuptools
2026-04-08 04:08:45.773905 | instance |  Downloading kubernetes
2026-04-08 04:08:45.801479 | instance |  Downloading ansible-core
2026-04-08 04:08:45.839094 | instance |  Downloading openstacksdk
2026-04-08 04:08:46.243998 | instance |       Built pyperclip==1.9.0
2026-04-08 04:08:46.369601 | instance |       Built atmosphere @ file:///home/zuul/src/github.com/vexxhost/atmosphere
2026-04-08 04:08:46.415096 | instance | Installed 83 packages in 41ms
2026-04-08 04:08:47.018169 | instance | WARNING  Molecule scenarios should migrate to 'extensions/molecule'
2026-04-08 04:08:47.617256 | instance | INFO     [aio > discovery] scenario test matrix: dependency, cleanup, destroy, syntax, create, prepare, converge, idempotence, side_effect, verify, cleanup, destroy
2026-04-08 04:08:47.617325 | instance | INFO     [aio > prerun] Performing prerun with role_name_check=0...
2026-04-08 04:10:00.926863 | instance | INFO     [aio > dependency] Executing
2026-04-08 04:10:00.927178 | instance | WARNING  [aio > dependency] Missing roles requirements file: requirements.yml
2026-04-08 04:10:00.927473 | instance | WARNING  [aio > dependency] Missing collections requirements file: collections.yml
2026-04-08 04:10:00.927616 | instance | WARNING  [aio > dependency] Executed: 2 missing (Remove from test_sequence to suppress)
2026-04-08 04:10:00.937952 | instance | INFO     [aio > cleanup] Executing
2026-04-08 04:10:00.938268 | instance | WARNING  [aio > cleanup] Executed: Missing playbook (Remove from test_sequence to suppress)
2026-04-08 04:10:00.947918 | instance | INFO     [aio > destroy] Executing
2026-04-08 04:10:00.947954 | instance | WARNING  [aio > destroy] Skipping, '--destroy=never' requested.
2026-04-08 04:10:00.948022 | instance | INFO     [aio > destroy] Executed: Successful
2026-04-08 04:10:00.957604 | instance | INFO     [aio > syntax] Executing
2026-04-08 04:10:03.752558 | instance |
2026-04-08 04:10:03.752825 | instance | playbook: /home/zuul/src/github.com/vexxhost/atmosphere/molecule/aio/converge.yml
2026-04-08 04:10:03.870084 | instance | INFO     [aio > syntax] Executed: Successful
2026-04-08 04:10:03.883606 | instance | INFO     [aio > create] Executing
2026-04-08 04:10:03.885893 | instance | WARNING  [aio > create] Executed: Missing playbook (Remove from test_sequence to suppress)
2026-04-08 04:10:03.896694 | instance | INFO     [aio > prepare] Executing
2026-04-08 04:10:04.731687 | instance |
2026-04-08 04:10:04.731872 | instance | PLAY [Prepare] *****************************************************************
2026-04-08 04:10:04.732168 | instance |
2026-04-08 04:10:04.732436 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 04:10:04.732706 | instance | Wednesday 08 April 2026  04:10:04 +0000 (0:00:00.033)       0:00:00.033 *******
2026-04-08 04:10:05.843085 | instance | [WARNING]: Platform linux on host instance is using the discovered Python
2026-04-08 04:10:05.843306 | instance | interpreter at /usr/bin/python3.10, but future installation of another Python
2026-04-08 04:10:05.843602 | instance | interpreter could change the meaning of that path. See
2026-04-08 04:10:05.843874 | instance | https://docs.ansible.com/ansible-
2026-04-08 04:10:05.844091 | instance | core/2.17/reference_appendices/interpreter_discovery.html for more information.
2026-04-08 04:10:05.853019 | instance | ok: [instance]
2026-04-08 04:10:05.853290 | instance |
2026-04-08 04:10:05.853529 | instance | TASK [Configure short hostname] ************************************************
2026-04-08 04:10:05.853806 | instance | Wednesday 08 April 2026  04:10:05 +0000 (0:00:01.122)       0:00:01.155 *******
2026-04-08 04:10:06.537238 | instance | changed: [instance]
2026-04-08 04:10:06.537418 | instance |
2026-04-08 04:10:06.537698 | instance | TASK [Ensure hostname inside hosts file] ***************************************
2026-04-08 04:10:06.537974 | instance | Wednesday 08 April 2026  04:10:06 +0000 (0:00:00.683)       0:00:01.839 *******
2026-04-08 04:10:06.790826 | instance | [WARNING]: Module remote_tmp /root/.ansible/tmp did not exist and was created
2026-04-08 04:10:06.791258 | instance | with a mode of 0700, this may cause issues when running as another user. To
2026-04-08 04:10:06.791549 | instance | avoid this, create the remote_tmp dir with the correct permissions manually
2026-04-08 04:10:06.802152 | instance | changed: [instance]
2026-04-08 04:10:06.802379 | instance |
2026-04-08 04:10:06.802655 | instance | TASK [Install "dirmngr" for GPG keyserver operations] **************************
2026-04-08 04:10:06.803026 | instance | Wednesday 08 April 2026  04:10:06 +0000 (0:00:00.265)       0:00:02.104 *******
2026-04-08 04:10:07.893719 | instance | ok: [instance]
2026-04-08 04:10:07.893900 | instance |
2026-04-08 04:10:07.894248 | instance | TASK [Purge "snapd" package] ***************************************************
2026-04-08 04:10:07.894561 | instance | Wednesday 08 April 2026  04:10:07 +0000 (0:00:01.091)       0:00:03.196 *******
2026-04-08 04:10:08.545874 | instance | ok: [instance]
2026-04-08 04:10:08.546062 | instance |
2026-04-08 04:10:08.546361 | instance | PLAY [Generate workspace for Atmosphere] ***************************************
2026-04-08 04:10:08.546625 | instance |
2026-04-08 04:10:08.547010 | instance | TASK [Create folders for workspace] ********************************************
2026-04-08 04:10:08.547344 | instance | Wednesday 08 April 2026  04:10:08 +0000 (0:00:00.651)       0:00:03.848 *******
2026-04-08 04:10:09.538631 | instance | changed: [localhost] => (item=group_vars)
2026-04-08 04:10:09.538759 | instance | changed: [localhost] => (item=group_vars/all)
2026-04-08 04:10:09.539096 | instance | changed: [localhost] => (item=group_vars/controllers)
2026-04-08 04:10:09.539369 | instance | changed: [localhost] => (item=group_vars/cephs)
2026-04-08 04:10:09.539633 | instance | changed: [localhost] => (item=group_vars/computes)
2026-04-08 04:10:09.539903 | instance | changed: [localhost] => (item=host_vars)
2026-04-08 04:10:09.540155 | instance |
2026-04-08 04:10:09.540459 | instance | PLAY [Generate Ceph control plane configuration for workspace] *****************
2026-04-08 04:10:09.540712 | instance |
2026-04-08 04:10:09.540998 | instance | TASK [Ensure the Ceph control plane configuration file exists] *****************
2026-04-08 04:10:09.541257 | instance | Wednesday 08 April 2026  04:10:09 +0000 (0:00:00.993)       0:00:04.841 *******
2026-04-08 04:10:09.713958 | instance | changed: [localhost]
2026-04-08 04:10:09.714224 | instance |
2026-04-08 04:10:09.714504 | instance | TASK [Load the current Ceph control plane configuration into a variable] *******
2026-04-08 04:10:09.714776 | instance | Wednesday 08 April 2026  04:10:09 +0000 (0:00:00.175)       0:00:05.016 *******
2026-04-08 04:10:09.746708 | instance | ok: [localhost]
2026-04-08 04:10:09.747022 | instance |
2026-04-08 04:10:09.747301 | instance | TASK [Generate Ceph control plane values for missing variables] ****************
2026-04-08 04:10:09.747570 | instance | Wednesday 08 April 2026  04:10:09 +0000 (0:00:00.032)       0:00:05.049 *******
2026-04-08 04:10:09.794949 | instance | ok: [localhost] => (item={'key': 'ceph_fsid', 'value': '5ada2aa1-30ab-5356-92f5-2e533f55651d'})
2026-04-08 04:10:09.795228 | instance | ok: [localhost] => (item={'key': 'ceph_mon_public_network', 'value': '10.96.240.0/24'})
2026-04-08 04:10:09.795549 | instance |
2026-04-08 04:10:09.795824 | instance | TASK [Write new Ceph control plane configuration file to disk] *****************
2026-04-08 04:10:09.796089 | instance | Wednesday 08 April 2026  04:10:09 +0000 (0:00:00.048)       0:00:05.097 *******
2026-04-08 04:10:10.292044 | instance | changed: [localhost]
2026-04-08 04:10:10.292284 | instance |
2026-04-08 04:10:10.292593 | instance | PLAY [Generate Ceph OSD configuration for workspace] ***************************
2026-04-08 04:10:10.292910 | instance |
2026-04-08 04:10:10.293228 | instance | TASK [Ensure the Ceph OSDs configuration file exists] **************************
2026-04-08 04:10:10.293522 | instance | Wednesday 08 April 2026  04:10:10 +0000 (0:00:00.496)       0:00:05.594 *******
2026-04-08 04:10:10.482217 | instance | changed: [localhost]
2026-04-08 04:10:10.482284 | instance |
2026-04-08 04:10:10.482298 | instance | TASK [Load the current Ceph OSDs configuration into a variable] ****************
2026-04-08 04:10:10.482487 | instance | Wednesday 08 April 2026  04:10:10 +0000 (0:00:00.188)       0:00:05.783 *******
2026-04-08 04:10:10.510852 | instance | ok: [localhost]
2026-04-08 04:10:10.511083 | instance |
2026-04-08 04:10:10.511398 | instance | TASK [Generate Ceph OSDs values for missing variables] *************************
2026-04-08 04:10:10.511671 | instance | Wednesday 08 April 2026  04:10:10 +0000 (0:00:00.030)       0:00:05.813 *******
2026-04-08 04:10:10.543329 | instance | ok: [localhost] => (item={'key': 'ceph_osd_devices', 'value': ['/dev/vdb', '/dev/vdc', '/dev/vdd']})
2026-04-08 04:10:10.543571 | instance |
2026-04-08 04:10:10.543845 | instance | TASK [Write new Ceph OSDs configuration file to disk] **************************
2026-04-08 04:10:10.544113 | instance | Wednesday 08 April 2026  04:10:10 +0000 (0:00:00.032)       0:00:05.846 *******
2026-04-08 04:10:10.876304 | instance | changed: [localhost]
2026-04-08 04:10:10.876492 | instance |
2026-04-08 04:10:10.876762 | instance | PLAY [Generate Kubernetes configuration for workspace] *************************
2026-04-08 04:10:10.877012 | instance |
2026-04-08 04:10:10.877284 | instance | TASK [Ensure the Kubernetes configuration file exists] *************************
2026-04-08 04:10:10.877560 | instance | Wednesday 08 April 2026  04:10:10 +0000 (0:00:00.332)       0:00:06.178 *******
2026-04-08 04:10:11.057980 | instance | changed: [localhost]
2026-04-08 04:10:11.058246 | instance |
2026-04-08 04:10:11.058488 | instance | TASK [Load the current Kubernetes configuration into a variable] ***************
2026-04-08 04:10:11.058770 | instance | Wednesday 08 April 2026  04:10:11 +0000 (0:00:00.180)       0:00:06.358 *******
2026-04-08 04:10:11.085709 | instance | ok: [localhost]
2026-04-08 04:10:11.085965 | instance |
2026-04-08 04:10:11.086248 | instance | TASK [Generate Kubernetes values for missing variables] ************************
2026-04-08 04:10:11.086530 | instance | Wednesday 08 April 2026  04:10:11 +0000 (0:00:00.029)       0:00:06.388 *******
2026-04-08 04:10:11.123099 | instance | ok: [localhost] => (item={'key': 'kubernetes_hostname', 'value': '10.96.240.10'})
2026-04-08 04:10:11.123370 | instance | ok: [localhost] => (item={'key': 'kubernetes_keepalived_vrid', 'value': 42})
2026-04-08 04:10:11.123674 | instance | ok: [localhost] => (item={'key': 'kubernetes_keepalived_vip', 'value': '10.96.240.10'})
2026-04-08 04:10:11.123934 | instance |
2026-04-08 04:10:11.124153 | instance | TASK [Write new Kubernetes configuration file to disk] *************************
2026-04-08 04:10:11.124441 | instance | Wednesday 08 April 2026  04:10:11 +0000 (0:00:00.037)       0:00:06.425 *******
2026-04-08 04:10:11.450696 | instance | changed: [localhost]
2026-04-08 04:10:11.450965 | instance |
2026-04-08 04:10:11.451245 | instance | PLAY [Generate Keepalived configuration for workspace] *************************
2026-04-08 04:10:11.451497 | instance |
2026-04-08 04:10:11.451762 | instance | TASK [Ensure the Keeaplived configuration file exists] *************************
2026-04-08 04:10:11.452032 | instance | Wednesday 08 April 2026  04:10:11 +0000 (0:00:00.327)       0:00:06.753 *******
2026-04-08 04:10:11.613007 | instance | changed: [localhost]
2026-04-08 04:10:11.613254 | instance |
2026-04-08 04:10:11.613525 | instance | TASK [Load the current Keepalived configuration into a variable] ***************
2026-04-08 04:10:11.613793 | instance | Wednesday 08 April 2026  04:10:11 +0000 (0:00:00.162)       0:00:06.915 *******
2026-04-08 04:10:11.644621 | instance | ok: [localhost]
2026-04-08 04:10:11.644835 | instance |
2026-04-08 04:10:11.645114 | instance | TASK [Generate Keepalived values for missing variables] ************************
2026-04-08 04:10:11.645539 | instance | Wednesday 08 April 2026  04:10:11 +0000 (0:00:00.031)       0:00:06.947 *******
2026-04-08 04:10:11.680904 | instance | ok: [localhost] => (item={'key': 'keepalived_interface', 'value': 'br-ex'})
2026-04-08 04:10:11.681171 | instance | ok: [localhost] => (item={'key': 'keepalived_vip', 'value': '10.96.250.10'})
2026-04-08 04:10:11.681417 | instance |
2026-04-08 04:10:11.681688 | instance | TASK [Write new Keepalived configuration file to disk] *************************
2026-04-08 04:10:11.681965 | instance | Wednesday 08 April 2026  04:10:11 +0000 (0:00:00.036)       0:00:06.983 *******
2026-04-08 04:10:12.031681 | instance | changed: [localhost]
2026-04-08 04:10:12.031920 | instance |
2026-04-08 04:10:12.032249 | instance | PLAY [Generate endpoints for workspace] ****************************************
2026-04-08 04:10:12.032513 | instance |
2026-04-08 04:10:12.032795 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 04:10:12.033082 | instance | Wednesday 08 April 2026  04:10:12 +0000 (0:00:00.350)       0:00:07.334 *******
2026-04-08 04:10:12.692997 | instance | ok: [localhost]
2026-04-08 04:10:12.693178 | instance |
2026-04-08 04:10:12.693493 | instance | TASK [Ensure the endpoints file exists] ****************************************
2026-04-08 04:10:12.693763 | instance | Wednesday 08 April 2026  04:10:12 +0000 (0:00:00.661)       0:00:07.995 *******
2026-04-08 04:10:12.858053 | instance | changed: [localhost]
2026-04-08 04:10:12.858297 | instance |
2026-04-08 04:10:12.858570 | instance | TASK [Load the current endpoints into a variable] ******************************
2026-04-08 04:10:12.858836 | instance | Wednesday 08 April 2026  04:10:12 +0000 (0:00:00.165)       0:00:08.160 *******
2026-04-08 04:10:12.890289 | instance | ok: [localhost]
2026-04-08 04:10:12.890543 | instance |
2026-04-08 04:10:12.890816 | instance | TASK [Generate endpoint skeleton for missing variables] ************************
2026-04-08 04:10:12.891112 | instance | Wednesday 08 April 2026  04:10:12 +0000 (0:00:00.032)       0:00:08.193 *******
2026-04-08 04:10:13.629186 | instance | ok: [localhost] => (item=keycloak_host)
2026-04-08 04:10:13.629436 | instance | ok: [localhost] => (item=kube_prometheus_stack_grafana_host)
2026-04-08 04:10:13.629694 | instance | ok: [localhost] => (item=kube_prometheus_stack_alertmanager_host)
2026-04-08 04:10:13.629961 | instance | ok: [localhost] => (item=kube_prometheus_stack_prometheus_host)
2026-04-08 04:10:13.630226 | instance | ok: [localhost] => (item=openstack_helm_endpoints_region_name)
2026-04-08 04:10:13.630445 | instance | ok: [localhost] => (item=openstack_helm_endpoints_keystone_api_host)
2026-04-08 04:10:13.630707 | instance | ok: [localhost] => (item=openstack_helm_endpoints_glance_api_host)
2026-04-08 04:10:13.630992 | instance | ok: [localhost] => (item=openstack_helm_endpoints_cinder_api_host)
2026-04-08 04:10:13.631264 | instance | ok: [localhost] => (item=openstack_helm_endpoints_placement_api_host)
2026-04-08 04:10:13.631540 | instance | ok: [localhost] => (item=openstack_helm_endpoints_barbican_api_host)
2026-04-08 04:10:13.631869 | instance | ok: [localhost] => (item=openstack_helm_endpoints_neutron_api_host)
2026-04-08 04:10:13.632161 | instance | ok: [localhost] => (item=openstack_helm_endpoints_nova_api_host)
2026-04-08 04:10:13.632426 | instance | ok: [localhost] => (item=openstack_helm_endpoints_nova_novnc_host)
2026-04-08 04:10:13.632691 | instance | ok: [localhost] => (item=openstack_helm_endpoints_ironic_api_host)
2026-04-08 04:10:13.632956 | instance | ok: [localhost] => (item=openstack_helm_endpoints_designate_api_host)
2026-04-08 04:10:13.633177 | instance | ok: [localhost] => (item=openstack_helm_endpoints_octavia_api_host)
2026-04-08 04:10:13.633439 | instance | ok: [localhost] => (item=openstack_helm_endpoints_magnum_api_host)
2026-04-08 04:10:13.633753 | instance | ok: [localhost] => (item=openstack_helm_endpoints_magnum_registry_host)
2026-04-08 04:10:13.634015 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_api_host)
2026-04-08 04:10:13.634231 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_cfn_api_host)
2026-04-08 04:10:13.634491 | instance | ok: [localhost] => (item=openstack_helm_endpoints_horizon_api_host)
2026-04-08 04:10:13.634811 | instance | ok: [localhost] => (item=openstack_helm_endpoints_rgw_host)
2026-04-08 04:10:13.635111 | instance | ok: [localhost] => (item=openstack_helm_endpoints_manila_api_host)
2026-04-08 04:10:13.635366 | instance |
2026-04-08 04:10:13.635584 | instance | TASK [Write new endpoints file to disk] ****************************************
2026-04-08 04:10:13.635847 | instance | Wednesday 08 April 2026  04:10:13 +0000 (0:00:00.738)       0:00:08.931 *******
2026-04-08 04:10:13.961248 | instance | changed: [localhost]
2026-04-08 04:10:13.961482 | instance |
2026-04-08 04:10:13.961758 | instance | TASK [Ensure the endpoints file exists] ****************************************
2026-04-08 04:10:13.962042 | instance | Wednesday 08 April 2026  04:10:13 +0000 (0:00:00.332)       0:00:09.264 *******
2026-04-08 04:10:14.136329 | instance | changed: [localhost]
2026-04-08 04:10:14.136576 | instance |
2026-04-08 04:10:14.136852 | instance | PLAY [Generate Neutron configuration for workspace] ****************************
2026-04-08 04:10:14.137098 | instance |
2026-04-08 04:10:14.137364 | instance | TASK [Ensure the Neutron configuration file exists] ****************************
2026-04-08 04:10:14.137630 | instance | Wednesday 08 April 2026  04:10:14 +0000 (0:00:00.175)       0:00:09.439 *******
2026-04-08 04:10:14.309581 | instance | changed: [localhost]
2026-04-08 04:10:14.309760 | instance |
2026-04-08 04:10:14.309979 | instance | TASK [Load the current Neutron configuration into a variable] ******************
2026-04-08 04:10:14.310180 | instance | Wednesday 08 April 2026  04:10:14 +0000 (0:00:00.173)       0:00:09.612 *******
2026-04-08 04:10:14.344852 | instance | ok: [localhost]
2026-04-08 04:10:14.345010 | instance |
2026-04-08 04:10:14.345187 | instance | TASK [Generate Neutron values for missing variables] ***************************
2026-04-08 04:10:14.345364 | instance | Wednesday 08 April 2026  04:10:14 +0000 (0:00:00.035)       0:00:09.647 *******
2026-04-08 04:10:14.381934 | instance | ok: [localhost] => (item={'key': 'neutron_networks', 'value': [{'name': 'public', 'external': True, 'shared': True, 'mtu_size': 1500, 'port_security_enabled': True, 'provider_network_type': 'flat', 'provider_physical_network': 'external', 'subnets': [{'name': 'public-subnet', 'cidr': '10.96.250.0/24', 'gateway_ip': '10.96.250.10', 'allocation_pool_start': '10.96.250.200', 'allocation_pool_end': '10.96.250.220', 'enable_dhcp': True}]}]})
2026-04-08 04:10:14.382152 | instance |
2026-04-08 04:10:14.382432 | instance | TASK [Write new Neutron configuration file to disk] ****************************
2026-04-08 04:10:14.382712 | instance | Wednesday 08 April 2026  04:10:14 +0000 (0:00:00.036)       0:00:09.684 *******
2026-04-08 04:10:14.716138 | instance | changed: [localhost]
2026-04-08 04:10:14.716364 | instance |
2026-04-08 04:10:14.716629 | instance | PLAY [Generate Nova configuration for workspace] *******************************
2026-04-08 04:10:14.716869 | instance |
2026-04-08 04:10:14.717131 | instance | TASK [Ensure the Nova configuration file exists] *******************************
2026-04-08 04:10:14.717393 | instance | Wednesday 08 April 2026  04:10:14 +0000 (0:00:00.334)       0:00:10.018 *******
2026-04-08 04:10:14.888517 | instance | changed: [localhost]
2026-04-08 04:10:14.888769 | instance |
2026-04-08 04:10:14.889041 | instance | TASK [Load the current Nova configuration into a variable] *********************
2026-04-08 04:10:14.889300 | instance | Wednesday 08 April 2026  04:10:14 +0000 (0:00:00.172)       0:00:10.191 *******
2026-04-08 04:10:14.918151 | instance | ok: [localhost]
2026-04-08 04:10:14.918380 | instance |
2026-04-08 04:10:14.918650 | instance | TASK [Generate Nova values for missing variables] ******************************
2026-04-08 04:10:14.919075 | instance | Wednesday 08 April 2026  04:10:14 +0000 (0:00:00.029)       0:00:10.221 *******
2026-04-08 04:10:14.956852 | instance | ok: [localhost] => (item={'key': 'nova_flavors', 'value': [{'name': 'm1.tiny', 'ram': 512, 'disk': 1, 'vcpus': 1}, {'name': 'm1.small', 'ram': 2048, 'disk': 20, 'vcpus': 1}, {'name': 'm1.medium', 'ram': 4096, 'disk': 40, 'vcpus': 2}, {'name': 'm1.large', 'ram': 8192, 'disk': 80, 'vcpus': 4}, {'name': 'm1.xlarge', 'ram': 16384, 'disk': 160, 'vcpus': 8}]})
2026-04-08 04:10:14.957087 | instance |
2026-04-08 04:10:14.957349 | instance | TASK [Write new Nova configuration file to disk] *******************************
2026-04-08 04:10:14.957612 | instance | Wednesday 08 April 2026  04:10:14 +0000 (0:00:00.037)       0:00:10.259 *******
2026-04-08 04:10:15.292479 | instance | changed: [localhost]
2026-04-08 04:10:15.292651 | instance |
2026-04-08 04:10:15.293253 | instance | PLAY [Generate secrets for workspace] ******************************************
2026-04-08 04:10:15.293521 | instance |
2026-04-08 04:10:15.293811 | instance | TASK [Ensure the secrets file exists] ******************************************
2026-04-08 04:10:15.294138 | instance | Wednesday 08 April 2026  04:10:15 +0000 (0:00:00.335)       0:00:10.594 *******
2026-04-08 04:10:15.467966 | instance | changed: [localhost]
2026-04-08 04:10:15.468255 | instance |
2026-04-08 04:10:15.468549 | instance | TASK [Load the current secrets into a variable] ********************************
2026-04-08 04:10:15.468833 | instance | Wednesday 08 April 2026  04:10:15 +0000 (0:00:00.175)       0:00:10.769 *******
2026-04-08 04:10:15.500869 | instance | ok: [localhost]
2026-04-08 04:10:15.501120 | instance |
2026-04-08 04:10:15.501403 | instance | TASK [Generate secrets for missing variables] **********************************
2026-04-08 04:10:15.501677 | instance | Wednesday 08 April 2026  04:10:15 +0000 (0:00:00.033)       0:00:10.803 *******
2026-04-08 04:10:15.889528 | instance | ok: [localhost] => (item=heat_auth_encryption_key)
2026-04-08 04:10:15.889807 | instance | ok: [localhost] => (item=keepalived_password)
2026-04-08 04:10:15.890970 | instance | ok: [localhost] => (item=keycloak_admin_password)
2026-04-08 04:10:15.890991 | instance | ok: [localhost] => (item=keycloak_database_password)
2026-04-08 04:10:15.891001 | instance | ok: [localhost] => (item=keystone_keycloak_client_secret)
2026-04-08 04:10:15.891011 | instance | ok: [localhost] => (item=keystone_oidc_crypto_passphrase)
2026-04-08 04:10:15.891204 | instance | ok: [localhost] => (item=kube_prometheus_stack_grafana_admin_password)
2026-04-08 04:10:15.891468 | instance | ok: [localhost] => (item=octavia_heartbeat_key)
2026-04-08 04:10:15.891738 | instance | ok: [localhost] => (item=openstack_helm_endpoints_rabbitmq_admin_password)
2026-04-08 04:10:15.892030 | instance | ok: [localhost] => (item=openstack_helm_endpoints_memcached_secret_key)
2026-04-08 04:10:15.892298 | instance | ok: [localhost] => (item=openstack_helm_endpoints_keystone_admin_password)
2026-04-08 04:10:15.892564 | instance | ok: [localhost] => (item=openstack_helm_endpoints_keystone_mariadb_password)
2026-04-08 04:10:15.892832 | instance | ok: [localhost] => (item=openstack_helm_endpoints_keystone_rabbitmq_password)
2026-04-08 04:10:15.893094 | instance | ok: [localhost] => (item=openstack_helm_endpoints_glance_keystone_password)
2026-04-08 04:10:15.893358 | instance | ok: [localhost] => (item=openstack_helm_endpoints_glance_mariadb_password)
2026-04-08 04:10:15.893624 | instance | ok: [localhost] => (item=openstack_helm_endpoints_glance_rabbitmq_password)
2026-04-08 04:10:15.893887 | instance | ok: [localhost] => (item=openstack_helm_endpoints_cinder_keystone_password)
2026-04-08 04:10:15.894159 | instance | ok: [localhost] => (item=openstack_helm_endpoints_cinder_mariadb_password)
2026-04-08 04:10:15.894428 | instance | ok: [localhost] => (item=openstack_helm_endpoints_cinder_rabbitmq_password)
2026-04-08 04:10:15.894690 | instance | ok: [localhost] => (item=openstack_helm_endpoints_placement_keystone_password)
2026-04-08 04:10:15.894983 | instance | ok: [localhost] => (item=openstack_helm_endpoints_placement_mariadb_password)
2026-04-08 04:10:15.895247 | instance | ok: [localhost] => (item=openstack_helm_endpoints_barbican_keystone_password)
2026-04-08 04:10:15.895511 | instance | ok: [localhost] => (item=openstack_helm_endpoints_barbican_mariadb_password)
2026-04-08 04:10:15.895773 | instance | ok: [localhost] => (item=openstack_helm_endpoints_neutron_keystone_password)
2026-04-08 04:10:15.896085 | instance | ok: [localhost] => (item=openstack_helm_endpoints_neutron_mariadb_password)
2026-04-08 04:10:15.896274 | instance | ok: [localhost] => (item=openstack_helm_endpoints_neutron_rabbitmq_password)
2026-04-08 04:10:15.896438 | instance | ok: [localhost] => (item=openstack_helm_endpoints_neutron_metadata_secret)
2026-04-08 04:10:15.896603 | instance | ok: [localhost] => (item=openstack_helm_endpoints_nova_keystone_password)
2026-04-08 04:10:15.896766 | instance | ok: [localhost] => (item=openstack_helm_endpoints_nova_mariadb_password)
2026-04-08 04:10:15.896933 | instance | ok: [localhost] => (item=openstack_helm_endpoints_nova_rabbitmq_password)
2026-04-08 04:10:15.897100 | instance | ok: [localhost] => (item=openstack_helm_endpoints_ironic_keystone_password)
2026-04-08 04:10:15.897261 | instance | ok: [localhost] => (item=openstack_helm_endpoints_ironic_mariadb_password)
2026-04-08 04:10:15.897426 | instance | ok: [localhost] => (item=openstack_helm_endpoints_ironic_rabbitmq_password)
2026-04-08 04:10:15.897594 | instance | ok: [localhost] => (item=openstack_helm_endpoints_designate_keystone_password)
2026-04-08 04:10:15.897755 | instance | ok: [localhost] => (item=openstack_helm_endpoints_designate_mariadb_password)
2026-04-08 04:10:15.897920 | instance | ok: [localhost] => (item=openstack_helm_endpoints_designate_rabbitmq_password)
2026-04-08 04:10:15.898096 | instance | ok: [localhost] => (item=openstack_helm_endpoints_octavia_keystone_password)
2026-04-08 04:10:15.898349 | instance | ok: [localhost] => (item=openstack_helm_endpoints_octavia_mariadb_password)
2026-04-08 04:10:15.898528 | instance | ok: [localhost] => (item=openstack_helm_endpoints_octavia_rabbitmq_password)
2026-04-08 04:10:15.898694 | instance | ok: [localhost] => (item=openstack_helm_endpoints_magnum_keystone_password)
2026-04-08 04:10:15.898865 | instance | ok: [localhost] => (item=openstack_helm_endpoints_magnum_mariadb_password)
2026-04-08 04:10:15.899050 | instance | ok: [localhost] => (item=openstack_helm_endpoints_magnum_rabbitmq_password)
2026-04-08 04:10:15.899214 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_keystone_password)
2026-04-08 04:10:15.899391 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_trustee_keystone_password)
2026-04-08 04:10:15.899558 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_stack_user_keystone_password)
2026-04-08 04:10:15.899723 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_mariadb_password)
2026-04-08 04:10:15.899898 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_rabbitmq_password)
2026-04-08 04:10:15.900068 | instance | ok: [localhost] => (item=openstack_helm_endpoints_horizon_mariadb_password)
2026-04-08 04:10:15.900231 | instance | ok: [localhost] => (item=openstack_helm_endpoints_tempest_keystone_password)
2026-04-08 04:10:15.900431 | instance | ok: [localhost] => (item=openstack_helm_endpoints_openstack_exporter_keystone_password)
2026-04-08 04:10:15.900609 | instance | ok: [localhost] => (item=openstack_helm_endpoints_rgw_keystone_password)
2026-04-08 04:10:15.900795 | instance | ok: [localhost] => (item=openstack_helm_endpoints_manila_keystone_password)
2026-04-08 04:10:15.900961 | instance | ok: [localhost] => (item=openstack_helm_endpoints_manila_mariadb_password)
2026-04-08 04:10:15.901139 | instance | ok: [localhost] => (item=openstack_helm_endpoints_staffeln_mariadb_password)
2026-04-08 04:10:15.901295 | instance |
2026-04-08 04:10:15.901461 | instance | TASK [Generate base64 encoded secrets] *****************************************
2026-04-08 04:10:15.901627 | instance | Wednesday 08 April 2026  04:10:15 +0000 (0:00:00.388)       0:00:11.192 *******
2026-04-08 04:10:15.942825 | instance | ok: [localhost] => (item=barbican_kek)
2026-04-08 04:10:15.943281 | instance |
2026-04-08 04:10:15.943577 | instance | TASK [Generate temporary files for generating keys for missing variables] ******
2026-04-08 04:10:15.943888 | instance | Wednesday 08 April 2026  04:10:15 +0000 (0:00:00.053)       0:00:11.245 *******
2026-04-08 04:10:16.345966 | instance | changed: [localhost] => (item=manila_ssh_key)
2026-04-08 04:10:16.346224 | instance | changed: [localhost] => (item=nova_ssh_key)
2026-04-08 04:10:16.346483 | instance |
2026-04-08 04:10:16.346750 | instance | TASK [Generate SSH keys for missing variables] *********************************
2026-04-08 04:10:16.347052 | instance | Wednesday 08 April 2026  04:10:16 +0000 (0:00:00.403)       0:00:11.648 *******
2026-04-08 04:10:19.859711 | instance | changed: [localhost] => (item=manila_ssh_key)
2026-04-08 04:10:19.860124 | instance | changed: [localhost] => (item=nova_ssh_key)
2026-04-08 04:10:19.860441 | instance |
2026-04-08 04:10:19.860768 | instance | TASK [Set values for SSH keys] *************************************************
2026-04-08 04:10:19.861091 | instance | Wednesday 08 April 2026  04:10:19 +0000 (0:00:03.513)       0:00:15.162 *******
2026-04-08 04:10:19.911171 | instance | ok: [localhost] => (item=manila_ssh_key)
2026-04-08 04:10:19.911413 | instance | ok: [localhost] => (item=nova_ssh_key)
2026-04-08 04:10:19.911628 | instance |
2026-04-08 04:10:19.912000 | instance | TASK [Delete the temporary files generated for SSH keys] ***********************
2026-04-08 04:10:19.912184 | instance | Wednesday 08 April 2026  04:10:19 +0000 (0:00:00.049)       0:00:15.212 *******
2026-04-08 04:10:20.227093 | instance | changed: [localhost] => (item=manila_ssh_key)
2026-04-08 04:10:20.227150 | instance | changed: [localhost] => (item=nova_ssh_key)
2026-04-08 04:10:20.227168 | instance |
2026-04-08 04:10:20.227511 | instance | TASK [Write new secrets file to disk] ******************************************
2026-04-08 04:10:20.227792 | instance | Wednesday 08 April 2026  04:10:20 +0000 (0:00:00.316)       0:00:15.529 *******
2026-04-08 04:10:20.563164 | instance | changed: [localhost]
2026-04-08 04:10:20.563406 | instance |
2026-04-08 04:10:20.563728 | instance | TASK [Encrypt secrets file with Vault password] ********************************
2026-04-08 04:10:20.564002 | instance | Wednesday 08 April 2026  04:10:20 +0000 (0:00:00.336)       0:00:15.865 *******
2026-04-08 04:10:20.595480 | instance | skipping: [localhost]
2026-04-08 04:10:20.595737 | instance |
2026-04-08 04:10:20.596005 | instance | PLAY [Overwrite OSD devices with LVM-backed paths] *****************************
2026-04-08 04:10:20.596252 | instance |
2026-04-08 04:10:20.596515 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 04:10:20.596784 | instance | Wednesday 08 April 2026  04:10:20 +0000 (0:00:00.032)       0:00:15.898 *******
2026-04-08 04:10:21.261657 | instance | ok: [instance]
2026-04-08 04:10:21.261909 | instance |
2026-04-08 04:10:21.262202 | instance | TASK [Overwrite existing osds.yml file] ****************************************
2026-04-08 04:10:21.262526 | instance | Wednesday 08 April 2026  04:10:21 +0000 (0:00:00.665)       0:00:16.564 *******
2026-04-08 04:10:21.690235 | instance | changed: [instance]
2026-04-08 04:10:21.690479 | instance |
2026-04-08 04:10:21.690754 | instance | PLAY [Setup networking] ********************************************************
2026-04-08 04:10:21.691038 | instance |
2026-04-08 04:10:21.691342 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 04:10:21.691678 | instance | Wednesday 08 April 2026  04:10:21 +0000 (0:00:00.428)       0:00:16.992 *******
2026-04-08 04:10:22.392357 | instance | ok: [instance]
2026-04-08 04:10:22.392737 | instance |
2026-04-08 04:10:22.393083 | instance | TASK [Create bridge for management network] ************************************
2026-04-08 04:10:22.393422 | instance | Wednesday 08 April 2026  04:10:22 +0000 (0:00:00.702)       0:00:17.695 *******
2026-04-08 04:10:22.727669 | instance | ok: [instance]
2026-04-08 04:10:22.728032 | instance |
2026-04-08 04:10:22.728369 | instance | TASK [Create fake interface for management bridge] *****************************
2026-04-08 04:10:22.728699 | instance | Wednesday 08 April 2026  04:10:22 +0000 (0:00:00.333)       0:00:18.028 *******
2026-04-08 04:10:22.919478 | instance | ok: [instance]
2026-04-08 04:10:22.919824 | instance |
2026-04-08 04:10:22.920161 | instance | TASK [Assign dummy interface to management bridge] *****************************
2026-04-08 04:10:22.920484 | instance | Wednesday 08 April 2026  04:10:22 +0000 (0:00:00.193)       0:00:18.222 *******
2026-04-08 04:10:23.099280 | instance | ok: [instance]
2026-04-08 04:10:23.099590 | instance |
2026-04-08 04:10:23.099752 | instance | TASK [Assign IP address for management bridge] *********************************
2026-04-08 04:10:23.099922 | instance | Wednesday 08 April 2026  04:10:23 +0000 (0:00:00.178)       0:00:18.400 *******
2026-04-08 04:10:23.282821 | instance | ok: [instance]
2026-04-08 04:10:23.282860 | instance |
2026-04-08 04:10:23.282906 | instance | TASK [Bring up interfaces] *****************************************************
2026-04-08 04:10:23.282919 | instance | Wednesday 08 April 2026  04:10:23 +0000 (0:00:00.183)       0:00:18.584 *******
2026-04-08 04:10:23.646186 | instance | ok: [instance] => (item=br-mgmt)
2026-04-08 04:10:23.646225 | instance | ok: [instance] => (item=dummy0)
2026-04-08 04:10:23.646231 | instance |
2026-04-08 04:10:23.646235 | instance | PLAY [Create devices for Ceph] *************************************************
2026-04-08 04:10:23.646240 | instance |
2026-04-08 04:10:23.646448 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 04:10:23.646725 | instance | Wednesday 08 April 2026  04:10:23 +0000 (0:00:00.363)       0:00:18.947 *******
2026-04-08 04:10:24.337423 | instance | ok: [instance]
2026-04-08 04:10:24.337672 | instance |
2026-04-08 04:10:24.337943 | instance | TASK [Install depedencies] *****************************************************
2026-04-08 04:10:24.338207 | instance | Wednesday 08 April 2026  04:10:24 +0000 (0:00:00.692)       0:00:19.640 *******
2026-04-08 04:10:45.067237 | instance | changed: [instance]
2026-04-08 04:10:45.067965 | instance |
2026-04-08 04:10:45.068023 | instance | TASK [Start up service] ********************************************************
2026-04-08 04:10:45.068051 | instance | Wednesday 08 April 2026  04:10:45 +0000 (0:00:20.729)       0:00:40.369 *******
2026-04-08 04:10:45.574336 | instance | ok: [instance]
2026-04-08 04:10:45.574431 | instance |
2026-04-08 04:10:45.574539 | instance | TASK [Generate lvm.conf] *******************************************************
2026-04-08 04:10:45.574726 | instance | Wednesday 08 April 2026  04:10:45 +0000 (0:00:00.506)       0:00:40.876 *******
2026-04-08 04:10:45.773748 | instance | ok: [instance]
2026-04-08 04:10:45.773902 | instance |
2026-04-08 04:10:45.774196 | instance | TASK [Write /etc/lvm/lvm.conf] *************************************************
2026-04-08 04:10:45.774434 | instance | Wednesday 08 April 2026  04:10:45 +0000 (0:00:00.199)       0:00:41.076 *******
2026-04-08 04:10:46.202982 | instance | changed: [instance]
2026-04-08 04:10:46.203048 | instance |
2026-04-08 04:10:46.203284 | instance | TASK [Get list of all loopback devices] ****************************************
2026-04-08 04:10:46.203326 | instance | Wednesday 08 April 2026  04:10:46 +0000 (0:00:00.429)       0:00:41.505 *******
2026-04-08 04:10:46.393255 | instance | ok: [instance]
2026-04-08 04:10:46.393391 | instance |
2026-04-08 04:10:46.393749 | instance | TASK [Fail if there is any existing loopback devices] **************************
2026-04-08 04:10:46.393786 | instance | Wednesday 08 April 2026  04:10:46 +0000 (0:00:00.190)       0:00:41.696 *******
2026-04-08 04:10:46.421383 | instance | skipping: [instance]
2026-04-08 04:10:46.421778 | instance |
2026-04-08 04:10:46.421804 | instance | TASK [Create devices for Ceph] *************************************************
2026-04-08 04:10:46.421810 | instance | Wednesday 08 April 2026  04:10:46 +0000 (0:00:00.028)       0:00:41.724 *******
2026-04-08 04:10:46.931256 | instance | changed: [instance] => (item=osd0)
2026-04-08 04:10:46.932070 | instance | changed: [instance] => (item=osd1)
2026-04-08 04:10:46.932110 | instance | changed: [instance] => (item=osd2)
2026-04-08 04:10:46.932118 | instance |
2026-04-08 04:10:46.932124 | instance | TASK [Set permissions on loopback devices] *************************************
2026-04-08 04:10:46.932131 | instance | Wednesday 08 April 2026  04:10:46 +0000 (0:00:00.509)       0:00:42.234 *******
2026-04-08 04:10:47.449200 | instance | changed: [instance] => (item=osd0)
2026-04-08 04:10:47.449263 | instance | changed: [instance] => (item=osd1)
2026-04-08 04:10:47.449271 | instance | changed: [instance] => (item=osd2)
2026-04-08 04:10:47.449277 | instance |
2026-04-08 04:10:47.449284 | instance | TASK [Start loop devices] ******************************************************
2026-04-08 04:10:47.449290 | instance | Wednesday 08 April 2026  04:10:47 +0000 (0:00:00.516)       0:00:42.751 *******
2026-04-08 04:10:48.152492 | instance | changed: [instance] => (item=osd0)
2026-04-08 04:10:48.152526 | instance | changed: [instance] => (item=osd1)
2026-04-08 04:10:48.152531 | instance | changed: [instance] => (item=osd2)
2026-04-08 04:10:48.152536 | instance |
2026-04-08 04:10:48.152541 | instance | TASK [Create a volume group for each loop device] ******************************
2026-04-08 04:10:48.152545 | instance | Wednesday 08 April 2026  04:10:48 +0000 (0:00:00.703)       0:00:43.455 *******
2026-04-08 04:10:51.228611 | instance | changed: [instance] => (item=osd0)
2026-04-08 04:10:51.229649 | instance | changed: [instance] => (item=osd1)
2026-04-08 04:10:51.229664 | instance | changed: [instance] => (item=osd2)
2026-04-08 04:10:51.229669 | instance |
2026-04-08 04:10:51.229674 | instance | TASK [Create a logical volume for each loop device] ****************************
2026-04-08 04:10:51.229679 | instance | Wednesday 08 April 2026  04:10:51 +0000 (0:00:03.075)       0:00:46.530 *******
2026-04-08 04:10:53.091750 | instance | changed: [instance] => (item=ceph-instance-osd0)
2026-04-08 04:10:53.091855 | instance | changed: [instance] => (item=ceph-instance-osd1)
2026-04-08 04:10:53.092590 | instance | changed: [instance] => (item=ceph-instance-osd2)
2026-04-08 04:10:53.092627 | instance |
2026-04-08 04:10:53.092632 | instance | PLAY [controllers] *************************************************************
2026-04-08 04:10:53.092637 | instance |
2026-04-08 04:10:53.092641 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 04:10:53.092646 | instance | Wednesday 08 April 2026  04:10:53 +0000 (0:00:01.863)       0:00:48.394 *******
2026-04-08 04:10:53.952494 | instance | ok: [instance]
2026-04-08 04:10:53.952603 | instance |
2026-04-08 04:10:53.952973 | instance | TASK [Set masquerade rule] *****************************************************
2026-04-08 04:10:53.953181 | instance | Wednesday 08 April 2026  04:10:53 +0000 (0:00:00.860)       0:00:49.255 *******
2026-04-08 04:10:54.393339 | instance | changed: [instance]
2026-04-08 04:10:54.394943 | instance |
2026-04-08 04:10:54.394984 | instance | PLAY RECAP *********************************************************************
2026-04-08 04:10:54.394998 | instance | instance                   : ok=26   changed=11   unreachable=0    failed=0    skipped=1    rescued=0    ignored=0
2026-04-08 04:10:54.395005 | instance | localhost                  : ok=40   changed=21   unreachable=0    failed=0    skipped=1    rescued=0    ignored=0
2026-04-08 04:10:54.395010 | instance |
2026-04-08 04:10:54.395016 | instance | Wednesday 08 April 2026  04:10:54 +0000 (0:00:00.440)       0:00:49.696 *******
2026-04-08 04:10:54.395216 | instance | ===============================================================================
2026-04-08 04:10:54.395563 | instance | Install depedencies ---------------------------------------------------- 20.73s
2026-04-08 04:10:54.395889 | instance | Generate SSH keys for missing variables --------------------------------- 3.51s
2026-04-08 04:10:54.396214 | instance | Create a volume group for each loop device ------------------------------ 3.08s
2026-04-08 04:10:54.396540 | instance | Create a logical volume for each loop device ---------------------------- 1.86s
2026-04-08 04:10:54.396933 | instance | Gathering Facts --------------------------------------------------------- 1.12s
2026-04-08 04:10:54.397263 | instance | Install "dirmngr" for GPG keyserver operations -------------------------- 1.09s
2026-04-08 04:10:54.397531 | instance | Create folders for workspace -------------------------------------------- 0.99s
2026-04-08 04:10:54.397731 | instance | Gathering Facts --------------------------------------------------------- 0.86s
2026-04-08 04:10:54.397934 | instance | Generate endpoint skeleton for missing variables ------------------------ 0.74s
2026-04-08 04:10:54.398139 | instance | Start loop devices ------------------------------------------------------ 0.70s
2026-04-08 04:10:54.398342 | instance | Gathering Facts --------------------------------------------------------- 0.70s
2026-04-08 04:10:54.398545 | instance | Gathering Facts --------------------------------------------------------- 0.69s
2026-04-08 04:10:54.398750 | instance | Configure short hostname ------------------------------------------------ 0.68s
2026-04-08 04:10:54.398996 | instance | Gathering Facts --------------------------------------------------------- 0.67s
2026-04-08 04:10:54.399210 | instance | Gathering Facts --------------------------------------------------------- 0.66s
2026-04-08 04:10:54.399421 | instance | Purge "snapd" package --------------------------------------------------- 0.65s
2026-04-08 04:10:54.399628 | instance | Set permissions on loopback devices ------------------------------------- 0.52s
2026-04-08 04:10:54.399833 | instance | Create devices for Ceph ------------------------------------------------- 0.51s
2026-04-08 04:10:54.400037 | instance | Start up service -------------------------------------------------------- 0.51s
2026-04-08 04:10:54.400244 | instance | Write new Ceph control plane configuration file to disk ----------------- 0.50s
2026-04-08 04:10:54.462152 | instance | INFO     [aio > prepare] Executed: Successful
2026-04-08 04:10:54.472103 | instance | INFO     [aio > converge] Executing
2026-04-08 04:10:57.190245 | instance |
2026-04-08 04:10:57.190657 | instance | PLAY [all] *********************************************************************
2026-04-08 04:10:57.191015 | instance |
2026-04-08 04:10:57.191390 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 04:10:57.191731 | instance | Wednesday 08 April 2026  04:10:57 +0000 (0:00:00.020)       0:00:00.020 *******
2026-04-08 04:10:58.336508 | instance | [WARNING]: Platform linux on host instance is using the discovered Python
2026-04-08 04:10:58.336937 | instance | interpreter at /usr/bin/python3.10, but future installation of another Python
2026-04-08 04:10:58.337279 | instance | interpreter could change the meaning of that path. See
2026-04-08 04:10:58.337612 | instance | https://docs.ansible.com/ansible-
2026-04-08 04:10:58.337947 | instance | core/2.17/reference_appendices/interpreter_discovery.html for more information.
2026-04-08 04:10:58.344037 | instance | ok: [instance]
2026-04-08 04:10:58.344365 | instance |
2026-04-08 04:10:58.344742 | instance | TASK [Fail if atmosphere_ceph_enabled is set] **********************************
2026-04-08 04:10:58.345268 | instance | Wednesday 08 April 2026  04:10:58 +0000 (0:00:01.153)       0:00:01.174 *******
2026-04-08 04:10:58.380689 | instance | skipping: [instance]
2026-04-08 04:10:58.381068 | instance |
2026-04-08 04:10:58.381450 | instance | TASK [Set a fact with the "atmosphere_images" for other plays] *****************
2026-04-08 04:10:58.381825 | instance | Wednesday 08 April 2026  04:10:58 +0000 (0:00:00.036)       0:00:01.210 *******
2026-04-08 04:10:58.566254 | instance | ok: [instance]
2026-04-08 04:10:58.566588 | instance |
2026-04-08 04:10:58.566968 | instance | PLAY [Deploy Ceph monitors & managers] *****************************************
2026-04-08 04:10:58.567339 | instance |
2026-04-08 04:10:58.567688 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 04:10:58.568022 | instance | Wednesday 08 April 2026  04:10:58 +0000 (0:00:00.185)       0:00:01.396 *******
2026-04-08 04:10:59.423991 | instance | ok: [instance]
2026-04-08 04:10:59.424401 | instance |
2026-04-08 04:10:59.424759 | instance | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-04-08 04:10:59.425093 | instance | Wednesday 08 April 2026  04:10:59 +0000 (0:00:00.857)       0:00:02.253 *******
2026-04-08 04:10:59.703460 | instance | ok: [instance]
2026-04-08 04:10:59.703551 | instance |
2026-04-08 04:10:59.703703 | instance | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-04-08 04:10:59.703903 | instance | Wednesday 08 April 2026  04:10:59 +0000 (0:00:00.277)       0:00:02.531 *******
2026-04-08 04:10:59.747315 | instance | skipping: [instance]
2026-04-08 04:10:59.747445 | instance |
2026-04-08 04:10:59.747628 | instance | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] ***
2026-04-08 04:10:59.747843 | instance | Wednesday 08 April 2026  04:10:59 +0000 (0:00:00.045)       0:00:02.576 *******
2026-04-08 04:11:00.023651 | instance | changed: [instance]
2026-04-08 04:11:00.024019 | instance |
2026-04-08 04:11:00.024410 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-04-08 04:11:00.024754 | instance | Wednesday 08 April 2026  04:11:00 +0000 (0:00:00.276)       0:00:02.853 *******
2026-04-08 04:11:00.091453 | instance | ok: [instance] => {
2026-04-08 04:11:00.091532 | instance |     "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.0/runc.amd64"
2026-04-08 04:11:00.091656 | instance | }
2026-04-08 04:11:00.091832 | instance |
2026-04-08 04:11:00.091984 | instance | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-04-08 04:11:00.092132 | instance | Wednesday 08 April 2026  04:11:00 +0000 (0:00:00.066)       0:00:02.920 *******
2026-04-08 04:11:00.855421 | instance | changed: [instance]
2026-04-08 04:11:00.855485 | instance |
2026-04-08 04:11:00.855566 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-04-08 04:11:00.855750 | instance | Wednesday 08 April 2026  04:11:00 +0000 (0:00:00.762)       0:00:03.682 *******
2026-04-08 04:11:00.907443 | instance | skipping: [instance]
2026-04-08 04:11:00.907549 | instance |
2026-04-08 04:11:00.907728 | instance | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-04-08 04:11:00.907881 | instance | Wednesday 08 April 2026  04:11:00 +0000 (0:00:00.052)       0:00:03.734 *******
2026-04-08 04:11:00.959238 | instance | skipping: [instance]
2026-04-08 04:11:00.959402 | instance |
2026-04-08 04:11:00.959554 | instance | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-04-08 04:11:00.959708 | instance | Wednesday 08 April 2026  04:11:00 +0000 (0:00:00.051)       0:00:03.786 *******
2026-04-08 04:11:01.159365 | instance | ok: [instance]
2026-04-08 04:11:01.159476 | instance |
2026-04-08 04:11:01.159662 | instance | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-04-08 04:11:01.159814 | instance | Wednesday 08 April 2026  04:11:01 +0000 (0:00:00.201)       0:00:03.987 *******
2026-04-08 04:11:02.283741 | instance | ok: [instance]
2026-04-08 04:11:02.284113 | instance |
2026-04-08 04:11:02.284545 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-04-08 04:11:02.284883 | instance | Wednesday 08 April 2026  04:11:02 +0000 (0:00:01.123)       0:00:05.111 *******
2026-04-08 04:11:02.347418 | instance | ok: [instance] => {
2026-04-08 04:11:02.347531 | instance |     "msg": "https://github.com/containerd/containerd/releases/download/v2.2.0/containerd-2.2.0-linux-amd64.tar.gz"
2026-04-08 04:11:02.347727 | instance | }
2026-04-08 04:11:02.347868 | instance |
2026-04-08 04:11:02.348018 | instance | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-04-08 04:11:02.348198 | instance | Wednesday 08 April 2026  04:11:02 +0000 (0:00:00.062)       0:00:05.174 *******
2026-04-08 04:11:03.035400 | instance | changed: [instance]
2026-04-08 04:11:03.035463 | instance |
2026-04-08 04:11:03.035606 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-04-08 04:11:03.035804 | instance | Wednesday 08 April 2026  04:11:03 +0000 (0:00:00.689)       0:00:05.863 *******
2026-04-08 04:11:05.855540 | instance | changed: [instance]
2026-04-08 04:11:05.855612 | instance |
2026-04-08 04:11:05.855740 | instance | TASK [vexxhost.containers.containerd : Install SELinux packages] ***************
2026-04-08 04:11:05.855920 | instance | Wednesday 08 April 2026  04:11:05 +0000 (0:00:02.819)       0:00:08.683 *******
2026-04-08 04:11:05.891344 | instance | skipping: [instance]
2026-04-08 04:11:05.891515 | instance |
2026-04-08 04:11:05.891670 | instance | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] ***
2026-04-08 04:11:05.891821 | instance | Wednesday 08 April 2026  04:11:05 +0000 (0:00:00.036)       0:00:08.720 *******
2026-04-08 04:11:05.926564 | instance | skipping: [instance]
2026-04-08 04:11:05.926761 | instance |
2026-04-08 04:11:05.926958 | instance | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ********
2026-04-08 04:11:05.927125 | instance | Wednesday 08 April 2026  04:11:05 +0000 (0:00:00.033)       0:00:08.753 *******
2026-04-08 04:11:05.954932 | instance | skipping: [instance]
2026-04-08 04:11:05.955164 | instance |
2026-04-08 04:11:05.955376 | instance | TASK [vexxhost.containers.containerd : Install AppArmor packages] **************
2026-04-08 04:11:05.955586 | instance | Wednesday 08 April 2026  04:11:05 +0000 (0:00:00.031)       0:00:08.785 *******
2026-04-08 04:11:11.128410 | instance | changed: [instance]
2026-04-08 04:11:11.128795 | instance |
2026-04-08 04:11:11.129162 | instance | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] ***
2026-04-08 04:11:11.129510 | instance | Wednesday 08 April 2026  04:11:11 +0000 (0:00:05.173)       0:00:13.958 *******
2026-04-08 04:11:11.798359 | instance | changed: [instance]
2026-04-08 04:11:11.798747 | instance |
2026-04-08 04:11:11.799181 | instance | TASK [vexxhost.containers.containerd : Create folders for configuration] *******
2026-04-08 04:11:11.799586 | instance | Wednesday 08 April 2026  04:11:11 +0000 (0:00:00.670)       0:00:14.628 *******
2026-04-08 04:11:12.670859 | instance | changed: [instance] => (item={'path': '/etc/containerd'})
2026-04-08 04:11:12.671353 | instance | changed: [instance] => (item={'path': '/var/lib/containerd', 'mode': '0o700'})
2026-04-08 04:11:12.671781 | instance | changed: [instance] => (item={'path': '/run/containerd', 'mode': '0o711'})
2026-04-08 04:11:12.672145 | instance | changed: [instance] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'})
2026-04-08 04:11:12.672496 | instance | changed: [instance] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'})
2026-04-08 04:11:12.672815 | instance |
2026-04-08 04:11:12.673157 | instance | TASK [vexxhost.containers.containerd : Create containerd config file] **********
2026-04-08 04:11:12.673492 | instance | Wednesday 08 April 2026  04:11:12 +0000 (0:00:00.872)       0:00:15.500 *******
2026-04-08 04:11:13.199479 | instance | changed: [instance]
2026-04-08 04:11:13.199561 | instance |
2026-04-08 04:11:13.199664 | instance | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ********
2026-04-08 04:11:13.199884 | instance | Wednesday 08 April 2026  04:11:13 +0000 (0:00:00.518)       0:00:16.019 *******
2026-04-08 04:11:13.200030 | instance |
2026-04-08 04:11:13.200185 | instance | RUNNING HANDLER [vexxhost.containers.containerd : Reload systemd] **************
2026-04-08 04:11:13.200334 | instance | Wednesday 08 April 2026  04:11:13 +0000 (0:00:00.008)       0:00:16.028 *******
2026-04-08 04:11:14.147616 | instance | ok: [instance]
2026-04-08 04:11:14.147705 | instance |
2026-04-08 04:11:14.147860 | instance | RUNNING HANDLER [vexxhost.containers.containerd : Restart containerd] **********
2026-04-08 04:11:14.148290 | instance | Wednesday 08 April 2026  04:11:14 +0000 (0:00:00.946)       0:00:16.974 *******
2026-04-08 04:11:14.567473 | instance | changed: [instance]
2026-04-08 04:11:14.567559 | instance |
2026-04-08 04:11:14.567690 | instance | TASK [vexxhost.containers.containerd : Enable and start service] ***************
2026-04-08 04:11:14.567883 | instance | Wednesday 08 April 2026  04:11:14 +0000 (0:00:00.422)       0:00:17.396 *******
2026-04-08 04:11:15.089107 | instance | changed: [instance]
2026-04-08 04:11:15.089180 | instance |
2026-04-08 04:11:15.089438 | instance | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-04-08 04:11:15.089488 | instance | Wednesday 08 April 2026  04:11:15 +0000 (0:00:00.522)       0:00:17.919 *******
2026-04-08 04:11:15.276161 | instance | ok: [instance]
2026-04-08 04:11:15.276217 | instance |
2026-04-08 04:11:15.276475 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-04-08 04:11:15.276516 | instance | Wednesday 08 April 2026  04:11:15 +0000 (0:00:00.186)       0:00:18.106 *******
2026-04-08 04:11:15.335700 | instance | ok: [instance] => {
2026-04-08 04:11:15.336408 | instance |     "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz"
2026-04-08 04:11:15.336444 | instance | }
2026-04-08 04:11:15.336450 | instance |
2026-04-08 04:11:15.336455 | instance | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-04-08 04:11:15.336460 | instance | Wednesday 08 April 2026  04:11:15 +0000 (0:00:00.059)       0:00:18.165 *******
2026-04-08 04:11:16.085338 | instance | changed: [instance]
2026-04-08 04:11:16.085409 | instance |
2026-04-08 04:11:16.085741 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-04-08 04:11:16.085781 | instance | Wednesday 08 April 2026  04:11:16 +0000 (0:00:00.749)       0:00:18.915 *******
2026-04-08 04:11:20.285433 | instance | changed: [instance]
2026-04-08 04:11:20.285501 | instance |
2026-04-08 04:11:20.285753 | instance | TASK [vexxhost.containers.docker : Install AppArmor packages] ******************
2026-04-08 04:11:20.285788 | instance | Wednesday 08 April 2026  04:11:20 +0000 (0:00:04.200)       0:00:23.115 *******
2026-04-08 04:11:21.223157 | instance | ok: [instance]
2026-04-08 04:11:21.223229 | instance |
2026-04-08 04:11:21.223527 | instance | TASK [vexxhost.containers.docker : Ensure group "docker" exists] ***************
2026-04-08 04:11:21.223570 | instance | Wednesday 08 April 2026  04:11:21 +0000 (0:00:00.937)       0:00:24.053 *******
2026-04-08 04:11:21.613160 | instance | changed: [instance]
2026-04-08 04:11:21.613226 | instance |
2026-04-08 04:11:21.613475 | instance | TASK [vexxhost.containers.docker : Create systemd service file for docker] *****
2026-04-08 04:11:21.613508 | instance | Wednesday 08 April 2026  04:11:21 +0000 (0:00:00.390)       0:00:24.443 *******
2026-04-08 04:11:22.054039 | instance | changed: [instance]
2026-04-08 04:11:22.054120 | instance |
2026-04-08 04:11:22.054355 | instance | TASK [vexxhost.containers.docker : Create folders for configuration] ***********
2026-04-08 04:11:22.054389 | instance | Wednesday 08 April 2026  04:11:22 +0000 (0:00:00.440)       0:00:24.884 *******
2026-04-08 04:11:22.597616 | instance | changed: [instance] => (item={'path': '/etc/docker'})
2026-04-08 04:11:22.597705 | instance | changed: [instance] => (item={'path': '/var/lib/docker', 'mode': '0o710'})
2026-04-08 04:11:22.598195 | instance | changed: [instance] => (item={'path': '/run/docker', 'mode': '0o711'})
2026-04-08 04:11:22.598230 | instance |
2026-04-08 04:11:22.598236 | instance | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ******
2026-04-08 04:11:22.598241 | instance | Wednesday 08 April 2026  04:11:22 +0000 (0:00:00.543)       0:00:25.427 *******
2026-04-08 04:11:23.048332 | instance | changed: [instance]
2026-04-08 04:11:23.048400 | instance |
2026-04-08 04:11:23.048652 | instance | TASK [vexxhost.containers.docker : Create docker daemon config file] ***********
2026-04-08 04:11:23.048690 | instance | Wednesday 08 April 2026  04:11:23 +0000 (0:00:00.450)       0:00:25.878 *******
2026-04-08 04:11:23.506403 | instance | changed: [instance]
2026-04-08 04:11:23.506472 | instance |
2026-04-08 04:11:23.506994 | instance | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************
2026-04-08 04:11:23.507054 | instance | Wednesday 08 April 2026  04:11:23 +0000 (0:00:00.447)       0:00:26.325 *******
2026-04-08 04:11:23.507066 | instance |
2026-04-08 04:11:23.507082 | instance | RUNNING HANDLER [vexxhost.containers.containerd : Reload systemd] **************
2026-04-08 04:11:23.507108 | instance | Wednesday 08 April 2026  04:11:23 +0000 (0:00:00.010)       0:00:26.336 *******
2026-04-08 04:11:24.248612 | instance | ok: [instance]
2026-04-08 04:11:24.248720 | instance |
2026-04-08 04:11:24.248806 | instance | RUNNING HANDLER [vexxhost.containers.docker : Restart docker] ******************
2026-04-08 04:11:24.248934 | instance | Wednesday 08 April 2026  04:11:24 +0000 (0:00:00.742)       0:00:27.078 *******
2026-04-08 04:11:25.527936 | instance | changed: [instance]
2026-04-08 04:11:25.528002 | instance |
2026-04-08 04:11:25.528303 | instance | TASK [vexxhost.containers.docker : Enable and start service] *******************
2026-04-08 04:11:25.528357 | instance | Wednesday 08 April 2026  04:11:25 +0000 (0:00:01.278)       0:00:28.357 *******
2026-04-08 04:11:26.082687 | instance | changed: [instance]
2026-04-08 04:11:26.083799 | instance |
2026-04-08 04:11:26.083841 | instance | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ******
2026-04-08 04:11:26.083854 | instance | Wednesday 08 April 2026  04:11:26 +0000 (0:00:00.555)       0:00:28.912 *******
2026-04-08 04:11:26.144590 | instance | ok: [instance] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/ubuntu-22.04.yml)
2026-04-08 04:11:26.144631 | instance |
2026-04-08 04:11:26.144819 | instance | TASK [vexxhost.ceph.cephadm : Install packages] ********************************
2026-04-08 04:11:26.145078 | instance | Wednesday 08 April 2026  04:11:26 +0000 (0:00:00.061)       0:00:28.974 *******
2026-04-08 04:11:30.868968 | instance | changed: [instance]
2026-04-08 04:11:30.869047 | instance |
2026-04-08 04:11:30.869114 | instance | TASK [vexxhost.ceph.cephadm : Ensure services are started] *********************
2026-04-08 04:11:30.869270 | instance | Wednesday 08 April 2026  04:11:30 +0000 (0:00:04.724)       0:00:33.699 *******
2026-04-08 04:11:31.502200 | instance | ok: [instance] => (item=chronyd)
2026-04-08 04:11:31.502302 | instance | ok: [instance] => (item=sshd)
2026-04-08 04:11:31.502727 | instance |
2026-04-08 04:11:31.502763 | instance | TASK [vexxhost.ceph.cephadm : Download "cephadm"] ******************************
2026-04-08 04:11:31.502771 | instance | Wednesday 08 April 2026  04:11:31 +0000 (0:00:00.633)       0:00:34.332 *******
2026-04-08 04:11:31.789639 | instance | changed: [instance]
2026-04-08 04:11:31.789700 | instance |
2026-04-08 04:11:31.790001 | instance | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ********************
2026-04-08 04:11:31.790158 | instance | Wednesday 08 April 2026  04:11:31 +0000 (0:00:00.287)       0:00:34.619 *******
2026-04-08 04:11:31.991215 | instance | ok: [instance]
2026-04-08 04:11:31.991261 | instance |
2026-04-08 04:11:31.991546 | instance | TASK [vexxhost.ceph.cephadm : Ensure "cephadm" user is present] ****************
2026-04-08 04:11:31.991588 | instance | Wednesday 08 April 2026  04:11:31 +0000 (0:00:00.201)       0:00:34.821 *******
2026-04-08 04:11:32.439541 | instance | changed: [instance]
2026-04-08 04:11:32.439577 | instance |
2026-04-08 04:11:32.439582 | instance | TASK [vexxhost.ceph.cephadm : Allow "cephadm" user to have passwordless sudo] ***
2026-04-08 04:11:32.439587 | instance | Wednesday 08 April 2026  04:11:32 +0000 (0:00:00.447)       0:00:35.268 *******
2026-04-08 04:11:32.790195 | instance | changed: [instance]
2026-04-08 04:11:32.790274 | instance |
2026-04-08 04:11:32.790564 | instance | TASK [vexxhost.ceph.mon : Get `cephadm ls` status] *****************************
2026-04-08 04:11:32.790596 | instance | Wednesday 08 April 2026  04:11:32 +0000 (0:00:00.351)       0:00:35.620 *******
2026-04-08 04:11:34.413042 | instance | ok: [instance]
2026-04-08 04:11:34.413416 | instance |
2026-04-08 04:11:34.413454 | instance | TASK [vexxhost.ceph.mon : Parse the `cephadm ls` output] ***********************
2026-04-08 04:11:34.413460 | instance | Wednesday 08 April 2026  04:11:34 +0000 (0:00:01.622)       0:00:37.243 *******
2026-04-08 04:11:34.458787 | instance | ok: [instance]
2026-04-08 04:11:34.459003 | instance |
2026-04-08 04:11:34.459082 | instance | TASK [vexxhost.ceph.mon : Assimilate existing configs in `ceph.conf`] **********
2026-04-08 04:11:34.459187 | instance | Wednesday 08 April 2026  04:11:34 +0000 (0:00:00.045)       0:00:37.288 *******
2026-04-08 04:11:34.500291 | instance | skipping: [instance]
2026-04-08 04:11:34.500656 | instance |
2026-04-08 04:11:34.500688 | instance | TASK [vexxhost.ceph.mon : Adopt monitor to cluster] ****************************
2026-04-08 04:11:34.500694 | instance | Wednesday 08 April 2026  04:11:34 +0000 (0:00:00.041)       0:00:37.330 *******
2026-04-08 04:11:34.538565 | instance | skipping: [instance]
2026-04-08 04:11:34.539025 | instance |
2026-04-08 04:11:34.539089 | instance | TASK [vexxhost.ceph.mon : Adopt manager to cluster] ****************************
2026-04-08 04:11:34.539101 | instance | Wednesday 08 April 2026  04:11:34 +0000 (0:00:00.038)       0:00:37.368 *******
2026-04-08 04:11:34.575786 | instance | skipping: [instance]
2026-04-08 04:11:34.575834 | instance |
2026-04-08 04:11:34.576100 | instance | TASK [vexxhost.ceph.mon : Enable "cephadm" mgr module] *************************
2026-04-08 04:11:34.576137 | instance | Wednesday 08 April 2026  04:11:34 +0000 (0:00:00.037)       0:00:37.405 *******
2026-04-08 04:11:34.614922 | instance | skipping: [instance]
2026-04-08 04:11:34.615320 | instance |
2026-04-08 04:11:34.615353 | instance | TASK [vexxhost.ceph.mon : Set orchestrator backend to "cephadm"] ***************
2026-04-08 04:11:34.615359 | instance | Wednesday 08 April 2026  04:11:34 +0000 (0:00:00.039)       0:00:37.445 *******
2026-04-08 04:11:34.654689 | instance | skipping: [instance]
2026-04-08 04:11:34.654805 | instance |
2026-04-08 04:11:34.655017 | instance | TASK [vexxhost.ceph.mon : Use `cephadm` user for cephadm] **********************
2026-04-08 04:11:34.655172 | instance | Wednesday 08 April 2026  04:11:34 +0000 (0:00:00.039)       0:00:37.484 *******
2026-04-08 04:11:34.693669 | instance | skipping: [instance]
2026-04-08 04:11:34.693775 | instance |
2026-04-08 04:11:34.693955 | instance | TASK [vexxhost.ceph.mon : Generate "cephadm" key] ******************************
2026-04-08 04:11:34.694107 | instance | Wednesday 08 April 2026  04:11:34 +0000 (0:00:00.038)       0:00:37.523 *******
2026-04-08 04:11:34.735156 | instance | skipping: [instance]
2026-04-08 04:11:34.735300 | instance |
2026-04-08 04:11:34.735444 | instance | TASK [vexxhost.ceph.mon : Set Ceph Monitor IP address] *************************
2026-04-08 04:11:34.735581 | instance | Wednesday 08 April 2026  04:11:34 +0000 (0:00:00.041)       0:00:37.565 *******
2026-04-08 04:11:34.845870 | instance | ok: [instance]
2026-04-08 04:11:34.846105 | instance |
2026-04-08 04:11:34.846429 | instance | TASK [vexxhost.ceph.mon : Check if any node is bootstrapped] *******************
2026-04-08 04:11:34.846698 | instance | Wednesday 08 April 2026  04:11:34 +0000 (0:00:00.110)       0:00:37.675 *******
2026-04-08 04:11:35.043800 | instance | ok: [instance] => (item=instance)
2026-04-08 04:11:35.044060 | instance |
2026-04-08 04:11:35.044384 | instance | TASK [vexxhost.ceph.mon : Select pre-existing bootstrap node if exists] ********
2026-04-08 04:11:35.044712 | instance | Wednesday 08 April 2026  04:11:35 +0000 (0:00:00.197)       0:00:37.873 *******
2026-04-08 04:11:35.094207 | instance | ok: [instance]
2026-04-08 04:11:35.094512 | instance |
2026-04-08 04:11:35.094787 | instance | TASK [vexxhost.ceph.mon : Bootstrap cluster] ***********************************
2026-04-08 04:11:35.095092 | instance | Wednesday 08 April 2026  04:11:35 +0000 (0:00:00.050)       0:00:37.924 *******
2026-04-08 04:11:35.164317 | instance | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/mon/tasks/bootstrap-ceph.yml for instance
2026-04-08 04:11:35.164576 | instance |
2026-04-08 04:11:35.164847 | instance | TASK [vexxhost.ceph.mon : Generate temporary file for "ceph.conf"] *************
2026-04-08 04:11:35.165117 | instance | Wednesday 08 April 2026  04:11:35 +0000 (0:00:00.070)       0:00:37.994 *******
2026-04-08 04:11:35.488867 | instance | changed: [instance]
2026-04-08 04:11:35.489120 | instance |
2026-04-08 04:11:35.489405 | instance | TASK [vexxhost.ceph.mon : Include extra configuration values] ******************
2026-04-08 04:11:35.489679 | instance | Wednesday 08 April 2026  04:11:35 +0000 (0:00:00.324)       0:00:38.318 *******
2026-04-08 04:11:36.153340 | instance | changed: [instance] => (item={'option': 'mon allow pool size one', 'section': 'global', 'value': True})
2026-04-08 04:11:36.153653 | instance | changed: [instance] => (item={'option': 'osd crush chooseleaf type', 'section': 'global', 'value': 0})
2026-04-08 04:11:36.153956 | instance | changed: [instance] => (item={'option': 'auth allow insecure global id reclaim', 'section': 'mon', 'value': False})
2026-04-08 04:11:36.154320 | instance |
2026-04-08 04:11:36.154683 | instance | TASK [vexxhost.ceph.mon : Run Bootstrap coomand] *******************************
2026-04-08 04:11:36.155014 | instance | Wednesday 08 April 2026  04:11:36 +0000 (0:00:00.664)       0:00:38.983 *******
2026-04-08 04:13:41.077201 | instance | ok: [instance]
2026-04-08 04:13:41.077273 | instance |
2026-04-08 04:13:41.077285 | instance | TASK [vexxhost.ceph.mon : Remove temporary file for "ceph.conf"] ***************
2026-04-08 04:13:41.077295 | instance | Wednesday 08 April 2026  04:13:41 +0000 (0:02:04.922)       0:02:43.905 *******
2026-04-08 04:13:41.282740 | instance | changed: [instance]
2026-04-08 04:13:41.283014 | instance |
2026-04-08 04:13:41.283080 | instance | TASK [vexxhost.ceph.mon : Set bootstrap node] **********************************
2026-04-08 04:13:41.283134 | instance | Wednesday 08 April 2026  04:13:41 +0000 (0:00:00.206)       0:02:44.112 *******
2026-04-08 04:13:41.322752 | instance | ok: [instance]
2026-04-08 04:13:41.322973 | instance |
2026-04-08 04:13:41.323056 | instance | TASK [Install Ceph host] *******************************************************
2026-04-08 04:13:41.323156 | instance | Wednesday 08 April 2026  04:13:41 +0000 (0:00:00.039)       0:02:44.152 *******
2026-04-08 04:13:41.406036 | instance | included: vexxhost.ceph.cephadm_host for instance
2026-04-08 04:13:41.406128 | instance |
2026-04-08 04:13:41.406379 | instance | TASK [vexxhost.ceph.cephadm_host : Get public SSH key for "cephadm" user] ******
2026-04-08 04:13:41.406403 | instance | Wednesday 08 April 2026  04:13:41 +0000 (0:00:00.083)       0:02:44.236 *******
2026-04-08 04:13:42.894453 | instance | ok: [instance]
2026-04-08 04:13:42.894533 | instance |
2026-04-08 04:13:42.894988 | instance | TASK [vexxhost.ceph.cephadm_host : Set fact with public SSH key for "cephadm" user] ***
2026-04-08 04:13:42.895067 | instance | Wednesday 08 April 2026  04:13:42 +0000 (0:00:01.488)       0:02:45.724 *******
2026-04-08 04:13:42.952151 | instance | ok: [instance] => (item=instance)
2026-04-08 04:13:42.952531 | instance |
2026-04-08 04:13:42.952568 | instance | TASK [vexxhost.ceph.cephadm_host : Set authorized key for "cephadm"] ***********
2026-04-08 04:13:42.952574 | instance | Wednesday 08 April 2026  04:13:42 +0000 (0:00:00.057)       0:02:45.782 *******
2026-04-08 04:13:43.345265 | instance | ok: [instance]
2026-04-08 04:13:43.345789 | instance |
2026-04-08 04:13:43.345833 | instance | TASK [vexxhost.ceph.cephadm_host : Add new host to Ceph] ***********************
2026-04-08 04:13:43.345841 | instance | Wednesday 08 April 2026  04:13:43 +0000 (0:00:00.393)       0:02:46.175 *******
2026-04-08 04:13:45.215862 | instance | ok: [instance]
2026-04-08 04:13:45.215965 | instance |
2026-04-08 04:13:45.216206 | instance | TASK [vexxhost.ceph.mon : Configure "mon" label for monitors] ******************
2026-04-08 04:13:45.216223 | instance | Wednesday 08 April 2026  04:13:45 +0000 (0:00:01.870)       0:02:48.046 *******
2026-04-08 04:13:46.808422 | instance | ok: [instance]
2026-04-08 04:13:46.808488 | instance |
2026-04-08 04:13:46.808618 | instance | TASK [vexxhost.ceph.mon : Validate monitor exist] ******************************
2026-04-08 04:13:46.808742 | instance | Wednesday 08 April 2026  04:13:46 +0000 (0:00:01.592)       0:02:49.638 *******
2026-04-08 04:13:56.985636 | instance | ok: [instance]
2026-04-08 04:13:56.985713 | instance |
2026-04-08 04:13:56.985969 | instance | TASK [Install Ceph host] *******************************************************
2026-04-08 04:13:56.986006 | instance | Wednesday 08 April 2026  04:13:56 +0000 (0:00:10.177)       0:02:59.815 *******
2026-04-08 04:13:57.064782 | instance | included: vexxhost.ceph.cephadm_host for instance
2026-04-08 04:13:57.065141 | instance |
2026-04-08 04:13:57.065172 | instance | TASK [vexxhost.ceph.cephadm_host : Get public SSH key for "cephadm" user] ******
2026-04-08 04:13:57.065179 | instance | Wednesday 08 April 2026  04:13:57 +0000 (0:00:00.079)       0:02:59.894 *******
2026-04-08 04:13:57.118218 | instance | skipping: [instance]
2026-04-08 04:13:57.118690 | instance |
2026-04-08 04:13:57.118713 | instance | TASK [vexxhost.ceph.cephadm_host : Set fact with public SSH key for "cephadm" user] ***
2026-04-08 04:13:57.118721 | instance | Wednesday 08 April 2026  04:13:57 +0000 (0:00:00.053)       0:02:59.948 *******
2026-04-08 04:13:57.168547 | instance | skipping: [instance] => (item=instance)
2026-04-08 04:13:57.168686 | instance | skipping: [instance]
2026-04-08 04:13:57.168826 | instance |
2026-04-08 04:13:57.169176 | instance | TASK [vexxhost.ceph.cephadm_host : Set authorized key for "cephadm"] ***********
2026-04-08 04:13:57.169193 | instance | Wednesday 08 April 2026  04:13:57 +0000 (0:00:00.050)       0:02:59.999 *******
2026-04-08 04:13:57.413042 | instance | ok: [instance]
2026-04-08 04:13:57.413119 | instance |
2026-04-08 04:13:57.413498 | instance | TASK [vexxhost.ceph.cephadm_host : Add new host to Ceph] ***********************
2026-04-08 04:13:57.413551 | instance | Wednesday 08 April 2026  04:13:57 +0000 (0:00:00.243)       0:03:00.243 *******
2026-04-08 04:13:59.339188 | instance | ok: [instance]
2026-04-08 04:13:59.339277 | instance |
2026-04-08 04:13:59.339341 | instance | TASK [vexxhost.ceph.mgr : Configure "mgr" label for managers] ******************
2026-04-08 04:13:59.339476 | instance | Wednesday 08 April 2026  04:13:59 +0000 (0:00:01.926)       0:03:02.169 *******
2026-04-08 04:14:00.944437 | instance | ok: [instance]
2026-04-08 04:14:00.944512 | instance |
2026-04-08 04:14:00.944782 | instance | TASK [vexxhost.ceph.mgr : Validate manager exist] ******************************
2026-04-08 04:14:00.944857 | instance | Wednesday 08 April 2026  04:14:00 +0000 (0:00:01.604)       0:03:03.774 *******
2026-04-08 04:14:02.488562 | instance | ok: [instance]
2026-04-08 04:14:02.488694 | instance |
2026-04-08 04:14:02.488706 | instance | TASK [vexxhost.ceph.mgr : Enable the Ceph Manager prometheus module] ***********
2026-04-08 04:14:02.488864 | instance | Wednesday 08 April 2026  04:14:02 +0000 (0:00:01.544)       0:03:05.318 *******
2026-04-08 04:14:04.595611 | instance | ok: [instance]
2026-04-08 04:14:04.595695 | instance |
2026-04-08 04:14:04.595777 | instance | PLAY [Deploy Ceph OSDs] ********************************************************
2026-04-08 04:14:04.595896 | instance |
2026-04-08 04:14:04.596021 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 04:14:04.596133 | instance | Wednesday 08 April 2026  04:14:04 +0000 (0:00:02.107)       0:03:07.425 *******
2026-04-08 04:14:05.479766 | instance | ok: [instance]
2026-04-08 04:14:05.479839 | instance |
2026-04-08 04:14:05.479956 | instance | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-04-08 04:14:05.480078 | instance | Wednesday 08 April 2026  04:14:05 +0000 (0:00:00.883)       0:03:08.309 *******
2026-04-08 04:14:05.692150 | instance | ok: [instance]
2026-04-08 04:14:05.692211 | instance |
2026-04-08 04:14:05.692376 | instance | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-04-08 04:14:05.692459 | instance | Wednesday 08 April 2026  04:14:05 +0000 (0:00:00.212)       0:03:08.522 *******
2026-04-08 04:14:05.726692 | instance | skipping: [instance]
2026-04-08 04:14:05.726772 | instance |
2026-04-08 04:14:05.726930 | instance | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] ***
2026-04-08 04:14:05.727117 | instance | Wednesday 08 April 2026  04:14:05 +0000 (0:00:00.034)       0:03:08.557 *******
2026-04-08 04:14:05.936410 | instance | ok: [instance]
2026-04-08 04:14:05.936480 | instance |
2026-04-08 04:14:05.936618 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-04-08 04:14:05.936790 | instance | Wednesday 08 April 2026  04:14:05 +0000 (0:00:00.209)       0:03:08.766 *******
2026-04-08 04:14:05.992092 | instance | ok: [instance] => {
2026-04-08 04:14:05.992196 | instance |     "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.0/runc.amd64"
2026-04-08 04:14:05.992301 | instance | }
2026-04-08 04:14:05.992415 | instance |
2026-04-08 04:14:05.992540 | instance | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-04-08 04:14:05.992647 | instance | Wednesday 08 April 2026  04:14:05 +0000 (0:00:00.055)       0:03:08.821 *******
2026-04-08 04:14:06.280605 | instance | ok: [instance]
2026-04-08 04:14:06.280725 | instance |
2026-04-08 04:14:06.280899 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-04-08 04:14:06.281075 | instance | Wednesday 08 April 2026  04:14:06 +0000 (0:00:00.288)       0:03:09.110 *******
2026-04-08 04:14:06.325559 | instance | skipping: [instance]
2026-04-08 04:14:06.325595 | instance |
2026-04-08 04:14:06.325743 | instance | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-04-08 04:14:06.325909 | instance | Wednesday 08 April 2026  04:14:06 +0000 (0:00:00.045)       0:03:09.156 *******
2026-04-08 04:14:06.372578 | instance | skipping: [instance]
2026-04-08 04:14:06.372614 | instance |
2026-04-08 04:14:06.372804 | instance | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-04-08 04:14:06.372872 | instance | Wednesday 08 April 2026  04:14:06 +0000 (0:00:00.047)       0:03:09.203 *******
2026-04-08 04:14:06.570113 | instance | ok: [instance]
2026-04-08 04:14:06.570195 | instance |
2026-04-08 04:14:06.570320 | instance | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-04-08 04:14:06.570442 | instance | Wednesday 08 April 2026  04:14:06 +0000 (0:00:00.197)       0:03:09.400 *******
2026-04-08 04:14:07.545414 | instance | ok: [instance]
2026-04-08 04:14:07.545460 | instance |
2026-04-08 04:14:07.545604 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-04-08 04:14:07.545724 | instance | Wednesday 08 April 2026  04:14:07 +0000 (0:00:00.975)       0:03:10.375 *******
2026-04-08 04:14:07.609357 | instance | ok: [instance] => {
2026-04-08 04:14:07.609483 | instance |     "msg": "https://github.com/containerd/containerd/releases/download/v2.2.0/containerd-2.2.0-linux-amd64.tar.gz"
2026-04-08 04:14:07.609691 | instance | }
2026-04-08 04:14:07.609801 | instance |
2026-04-08 04:14:07.610012 | instance | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-04-08 04:14:07.610122 | instance | Wednesday 08 April 2026  04:14:07 +0000 (0:00:00.063)       0:03:10.439 *******
2026-04-08 04:14:07.940243 | instance | ok: [instance]
2026-04-08 04:14:07.940286 | instance |
2026-04-08 04:14:07.940485 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-04-08 04:14:07.940549 | instance | Wednesday 08 April 2026  04:14:07 +0000 (0:00:00.330)       0:03:10.770 *******
2026-04-08 04:14:09.842916 | instance | ok: [instance]
2026-04-08 04:14:09.842979 | instance |
2026-04-08 04:14:09.843125 | instance | TASK [vexxhost.containers.containerd : Install SELinux packages] ***************
2026-04-08 04:14:09.843288 | instance | Wednesday 08 April 2026  04:14:09 +0000 (0:00:01.902)       0:03:12.673 *******
2026-04-08 04:14:09.873016 | instance | skipping: [instance]
2026-04-08 04:14:09.873261 | instance |
2026-04-08 04:14:09.873396 | instance | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] ***
2026-04-08 04:14:09.873613 | instance | Wednesday 08 April 2026  04:14:09 +0000 (0:00:00.030)       0:03:12.703 *******
2026-04-08 04:14:09.909073 | instance | skipping: [instance]
2026-04-08 04:14:09.909119 | instance |
2026-04-08 04:14:09.909265 | instance | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ********
2026-04-08 04:14:09.909382 | instance | Wednesday 08 April 2026  04:14:09 +0000 (0:00:00.035)       0:03:12.739 *******
2026-04-08 04:14:09.943357 | instance | skipping: [instance]
2026-04-08 04:14:09.943420 | instance |
2026-04-08 04:14:09.943597 | instance | TASK [vexxhost.containers.containerd : Install AppArmor packages] **************
2026-04-08 04:14:09.943659 | instance | Wednesday 08 April 2026  04:14:09 +0000 (0:00:00.034)       0:03:12.773 *******
2026-04-08 04:14:10.921173 | instance | ok: [instance]
2026-04-08 04:14:10.921249 | instance |
2026-04-08 04:14:10.921357 | instance | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] ***
2026-04-08 04:14:10.921458 | instance | Wednesday 08 April 2026  04:14:10 +0000 (0:00:00.977)       0:03:13.751 *******
2026-04-08 04:14:11.343645 | instance | ok: [instance]
2026-04-08 04:14:11.343705 | instance |
2026-04-08 04:14:11.343858 | instance | TASK [vexxhost.containers.containerd : Create folders for configuration] *******
2026-04-08 04:14:11.343982 | instance | Wednesday 08 April 2026  04:14:11 +0000 (0:00:00.422)       0:03:14.173 *******
2026-04-08 04:14:12.193663 | instance | ok: [instance] => (item={'path': '/etc/containerd'})
2026-04-08 04:14:12.193740 | instance | ok: [instance] => (item={'path': '/var/lib/containerd', 'mode': '0o700'})
2026-04-08 04:14:12.193913 | instance | ok: [instance] => (item={'path': '/run/containerd', 'mode': '0o711'})
2026-04-08 04:14:12.194047 | instance | ok: [instance] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'})
2026-04-08 04:14:12.194186 | instance | ok: [instance] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'})
2026-04-08 04:14:12.194300 | instance |
2026-04-08 04:14:12.194438 | instance | TASK [vexxhost.containers.containerd : Create containerd config file] **********
2026-04-08 04:14:12.194539 | instance | Wednesday 08 April 2026  04:14:12 +0000 (0:00:00.849)       0:03:15.023 *******
2026-04-08 04:14:12.670803 | instance | ok: [instance]
2026-04-08 04:14:12.670921 | instance |
2026-04-08 04:14:12.671099 | instance | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ********
2026-04-08 04:14:12.671292 | instance | Wednesday 08 April 2026  04:14:12 +0000 (0:00:00.468)       0:03:15.491 *******
2026-04-08 04:14:12.671441 | instance |
2026-04-08 04:14:12.671602 | instance | TASK [vexxhost.containers.containerd : Enable and start service] ***************
2026-04-08 04:14:12.671759 | instance | Wednesday 08 April 2026  04:14:12 +0000 (0:00:00.008)       0:03:15.500 *******
2026-04-08 04:14:13.021756 | instance | ok: [instance]
2026-04-08 04:14:13.021830 | instance |
2026-04-08 04:14:13.021925 | instance | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-04-08 04:14:13.022044 | instance | Wednesday 08 April 2026  04:14:13 +0000 (0:00:00.351)       0:03:15.851 *******
2026-04-08 04:14:13.212892 | instance | ok: [instance]
2026-04-08 04:14:13.212957 | instance |
2026-04-08 04:14:13.213068 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-04-08 04:14:13.213180 | instance | Wednesday 08 April 2026  04:14:13 +0000 (0:00:00.191)       0:03:16.043 *******
2026-04-08 04:14:13.264877 | instance | ok: [instance] => {
2026-04-08 04:14:13.264962 | instance |     "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz"
2026-04-08 04:14:13.264982 | instance | }
2026-04-08 04:14:13.265120 | instance |
2026-04-08 04:14:13.265260 | instance | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-04-08 04:14:13.265354 | instance | Wednesday 08 April 2026  04:14:13 +0000 (0:00:00.051)       0:03:16.095 *******
2026-04-08 04:14:13.601757 | instance | ok: [instance]
2026-04-08 04:14:13.601829 | instance |
2026-04-08 04:14:13.601938 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-04-08 04:14:13.602057 | instance | Wednesday 08 April 2026  04:14:13 +0000 (0:00:00.336)       0:03:16.432 *******
2026-04-08 04:14:16.649982 | instance | ok: [instance]
2026-04-08 04:14:16.650145 | instance |
2026-04-08 04:14:16.650451 | instance | TASK [vexxhost.containers.docker : Install AppArmor packages] ******************
2026-04-08 04:14:16.650733 | instance | Wednesday 08 April 2026  04:14:16 +0000 (0:00:03.046)       0:03:19.478 *******
2026-04-08 04:14:17.631159 | instance | ok: [instance]
2026-04-08 04:14:17.631938 | instance |
2026-04-08 04:14:17.631958 | instance | TASK [vexxhost.containers.docker : Ensure group "docker" exists] ***************
2026-04-08 04:14:17.631976 | instance | Wednesday 08 April 2026  04:14:17 +0000 (0:00:00.982)       0:03:20.461 *******
2026-04-08 04:14:17.823215 | instance | ok: [instance]
2026-04-08 04:14:17.823360 | instance |
2026-04-08 04:14:17.823375 | instance | TASK [vexxhost.containers.docker : Create systemd service file for docker] *****
2026-04-08 04:14:17.823518 | instance | Wednesday 08 April 2026  04:14:17 +0000 (0:00:00.192)       0:03:20.653 *******
2026-04-08 04:14:18.236019 | instance | ok: [instance]
2026-04-08 04:14:18.236149 | instance |
2026-04-08 04:14:18.236274 | instance | TASK [vexxhost.containers.docker : Create folders for configuration] ***********
2026-04-08 04:14:18.236395 | instance | Wednesday 08 April 2026  04:14:18 +0000 (0:00:00.412)       0:03:21.065 *******
2026-04-08 04:14:18.760471 | instance | ok: [instance] => (item={'path': '/etc/docker'})
2026-04-08 04:14:18.761039 | instance | ok: [instance] => (item={'path': '/var/lib/docker', 'mode': '0o710'})
2026-04-08 04:14:18.761055 | instance | ok: [instance] => (item={'path': '/run/docker', 'mode': '0o711'})
2026-04-08 04:14:18.761060 | instance |
2026-04-08 04:14:18.761066 | instance | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ******
2026-04-08 04:14:18.761072 | instance | Wednesday 08 April 2026  04:14:18 +0000 (0:00:00.525)       0:03:21.590 *******
2026-04-08 04:14:19.172081 | instance | ok: [instance]
2026-04-08 04:14:19.172167 | instance |
2026-04-08 04:14:19.172307 | instance | TASK [vexxhost.containers.docker : Create docker daemon config file] ***********
2026-04-08 04:14:19.172423 | instance | Wednesday 08 April 2026  04:14:19 +0000 (0:00:00.411)       0:03:22.002 *******
2026-04-08 04:14:19.576420 | instance | ok: [instance]
2026-04-08 04:14:19.576497 | instance |
2026-04-08 04:14:19.576632 | instance | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************
2026-04-08 04:14:19.576750 | instance | Wednesday 08 April 2026  04:14:19 +0000 (0:00:00.395)       0:03:22.398 *******
2026-04-08 04:14:19.576863 | instance |
2026-04-08 04:14:19.576986 | instance | TASK [vexxhost.containers.docker : Enable and start service] *******************
2026-04-08 04:14:19.577107 | instance | Wednesday 08 April 2026  04:14:19 +0000 (0:00:00.008)       0:03:22.406 *******
2026-04-08 04:14:19.915473 | instance | ok: [instance]
2026-04-08 04:14:19.915539 | instance |
2026-04-08 04:14:19.915716 | instance | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ******
2026-04-08 04:14:19.915809 | instance | Wednesday 08 April 2026  04:14:19 +0000 (0:00:00.338)       0:03:22.745 *******
2026-04-08 04:14:19.975687 | instance | ok: [instance] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/ubuntu-22.04.yml)
2026-04-08 04:14:19.975807 | instance |
2026-04-08 04:14:19.975960 | instance | TASK [vexxhost.ceph.cephadm : Install packages] ********************************
2026-04-08 04:14:19.976115 | instance | Wednesday 08 April 2026  04:14:19 +0000 (0:00:00.060)       0:03:22.805 *******
2026-04-08 04:14:20.902050 | instance | ok: [instance]
2026-04-08 04:14:20.902118 | instance |
2026-04-08 04:14:20.902267 | instance | TASK [vexxhost.ceph.cephadm : Ensure services are started] *********************
2026-04-08 04:14:20.902403 | instance | Wednesday 08 April 2026  04:14:20 +0000 (0:00:00.926)       0:03:23.732 *******
2026-04-08 04:14:21.534332 | instance | ok: [instance] => (item=chronyd)
2026-04-08 04:14:21.534478 | instance | ok: [instance] => (item=sshd)
2026-04-08 04:14:21.534638 | instance |
2026-04-08 04:14:21.534807 | instance | TASK [vexxhost.ceph.cephadm : Download "cephadm"] ******************************
2026-04-08 04:14:21.535001 | instance | Wednesday 08 April 2026  04:14:21 +0000 (0:00:00.631)       0:03:24.364 *******
2026-04-08 04:14:21.810122 | instance | ok: [instance]
2026-04-08 04:14:21.810187 | instance |
2026-04-08 04:14:21.810317 | instance | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ********************
2026-04-08 04:14:21.810439 | instance | Wednesday 08 April 2026  04:14:21 +0000 (0:00:00.275)       0:03:24.640 *******
2026-04-08 04:14:22.009116 | instance | ok: [instance]
2026-04-08 04:14:22.009200 | instance |
2026-04-08 04:14:22.009324 | instance | TASK [vexxhost.ceph.cephadm : Ensure "cephadm" user is present] ****************
2026-04-08 04:14:22.009448 | instance | Wednesday 08 April 2026  04:14:22 +0000 (0:00:00.199)       0:03:24.839 *******
2026-04-08 04:14:22.231648 | instance | ok: [instance]
2026-04-08 04:14:22.231771 | instance |
2026-04-08 04:14:22.231941 | instance | TASK [vexxhost.ceph.cephadm : Allow "cephadm" user to have passwordless sudo] ***
2026-04-08 04:14:22.232081 | instance | Wednesday 08 April 2026  04:14:22 +0000 (0:00:00.222)       0:03:25.061 *******
2026-04-08 04:14:22.429667 | instance | ok: [instance]
2026-04-08 04:14:22.429728 | instance |
2026-04-08 04:14:22.429867 | instance | TASK [vexxhost.ceph.osd : Get monitor status] **********************************
2026-04-08 04:14:22.429985 | instance | Wednesday 08 April 2026  04:14:22 +0000 (0:00:00.197)       0:03:25.259 *******
2026-04-08 04:14:22.652215 | instance | ok: [instance] => (item=instance)
2026-04-08 04:14:22.652302 | instance |
2026-04-08 04:14:22.652435 | instance | TASK [vexxhost.ceph.osd : Select admin host] ***********************************
2026-04-08 04:14:22.652554 | instance | Wednesday 08 April 2026  04:14:22 +0000 (0:00:00.222)       0:03:25.482 *******
2026-04-08 04:14:22.698628 | instance | ok: [instance]
2026-04-08 04:14:22.698688 | instance |
2026-04-08 04:14:22.698809 | instance | TASK [vexxhost.ceph.osd : Get `cephadm ls` status] *****************************
2026-04-08 04:14:22.698949 | instance | Wednesday 08 April 2026  04:14:22 +0000 (0:00:00.046)       0:03:25.529 *******
2026-04-08 04:14:27.973225 | instance | ok: [instance]
2026-04-08 04:14:27.973690 | instance |
2026-04-08 04:14:27.973709 | instance | TASK [vexxhost.ceph.osd : Parse the `cephadm ls` output] ***********************
2026-04-08 04:14:27.973716 | instance | Wednesday 08 April 2026  04:14:27 +0000 (0:00:05.274)       0:03:30.803 *******
2026-04-08 04:14:28.023288 | instance | ok: [instance]
2026-04-08 04:14:28.023782 | instance |
2026-04-08 04:14:28.023824 | instance | TASK [Install Ceph host] *******************************************************
2026-04-08 04:14:28.023831 | instance | Wednesday 08 April 2026  04:14:28 +0000 (0:00:00.049)       0:03:30.853 *******
2026-04-08 04:14:28.088711 | instance | included: vexxhost.ceph.cephadm_host for instance
2026-04-08 04:14:28.088762 | instance |
2026-04-08 04:14:28.089044 | instance | TASK [vexxhost.ceph.cephadm_host : Get public SSH key for "cephadm" user] ******
2026-04-08 04:14:28.089081 | instance | Wednesday 08 April 2026  04:14:28 +0000 (0:00:00.065)       0:03:30.918 *******
2026-04-08 04:14:28.134958 | instance | skipping: [instance]
2026-04-08 04:14:28.135143 | instance |
2026-04-08 04:14:28.135339 | instance | TASK [vexxhost.ceph.cephadm_host : Set fact with public SSH key for "cephadm" user] ***
2026-04-08 04:14:28.135564 | instance | Wednesday 08 April 2026  04:14:28 +0000 (0:00:00.045)       0:03:30.964 *******
2026-04-08 04:14:28.181211 | instance | skipping: [instance] => (item=instance)
2026-04-08 04:14:28.181381 | instance | skipping: [instance]
2026-04-08 04:14:28.181549 | instance |
2026-04-08 04:14:28.181717 | instance | TASK [vexxhost.ceph.cephadm_host : Set authorized key for "cephadm"] ***********
2026-04-08 04:14:28.181883 | instance | Wednesday 08 April 2026  04:14:28 +0000 (0:00:00.046)       0:03:31.011 *******
2026-04-08 04:14:28.430652 | instance | ok: [instance]
2026-04-08 04:14:28.430719 | instance |
2026-04-08 04:14:28.430936 | instance | TASK [vexxhost.ceph.cephadm_host : Add new host to Ceph] ***********************
2026-04-08 04:14:28.431125 | instance | Wednesday 08 April 2026  04:14:28 +0000 (0:00:00.249)       0:03:31.260 *******
2026-04-08 04:14:30.394711 | instance | ok: [instance]
2026-04-08 04:14:30.395033 | instance |
2026-04-08 04:14:30.395331 | instance | TASK [vexxhost.ceph.osd : Adopt OSDs to cluster] *******************************
2026-04-08 04:14:30.395649 | instance | Wednesday 08 April 2026  04:14:30 +0000 (0:00:01.963)       0:03:33.224 *******
2026-04-08 04:14:30.419682 | instance | skipping: [instance]
2026-04-08 04:14:30.419836 | instance |
2026-04-08 04:14:30.420016 | instance | TASK [vexxhost.ceph.osd : Wait until OSD added to cephadm] *********************
2026-04-08 04:14:30.420196 | instance | Wednesday 08 April 2026  04:14:30 +0000 (0:00:00.025)       0:03:33.249 *******
2026-04-08 04:14:30.446180 | instance | skipping: [instance]
2026-04-08 04:14:30.446404 | instance |
2026-04-08 04:14:30.446682 | instance | TASK [vexxhost.ceph.osd : Ensure all OSDs are non-legacy] **********************
2026-04-08 04:14:30.447055 | instance | Wednesday 08 April 2026  04:14:30 +0000 (0:00:00.026)       0:03:33.276 *******
2026-04-08 04:14:35.791004 | instance | ok: [instance]
2026-04-08 04:14:35.791065 | instance |
2026-04-08 04:14:35.791077 | instance | TASK [vexxhost.ceph.osd : Get `ceph-volume lvm list` status] *******************
2026-04-08 04:14:35.791092 | instance | Wednesday 08 April 2026  04:14:35 +0000 (0:00:05.343)       0:03:38.620 *******
2026-04-08 04:14:46.182625 | instance | ok: [instance]
2026-04-08 04:14:46.182716 | instance |
2026-04-08 04:14:46.183098 | instance | TASK [vexxhost.ceph.osd : Install OSDs] ****************************************
2026-04-08 04:14:46.183119 | instance | Wednesday 08 April 2026  04:14:46 +0000 (0:00:10.392)       0:03:49.012 *******
2026-04-08 04:14:53.811500 | instance | failed: [instance] (item=/dev/vdb) => {"ansible_loop_var": "item", "changed": false, "cmd": ["cephadm", "shell", "--fsid", "4837cbf8-4f90-4300-b3f6-726c9b9f89b4", "--config", "/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config", "--", "ceph", "orch", "daemon", "add", "osd", "instance:/dev/vdb"], "delta": "0:00:07.431833", "end": "2026-04-08 04:14:53.781674", "item": "/dev/vdb", "msg": "non-zero return code", "rc": 22, "start": "2026-04-08 04:14:46.349841", "stderr": "Error EINVAL: Traceback (most recent call last):\n  File \"/usr/share/ceph/mgr/mgr_module.py\", line 1834, in _handle_command\n    return self.handle_command(inbuf, cmd)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 183, in handle_command\n    return dispatch[cmd['prefix']].call(self, cmd, inbuf)\n  File \"/usr/share/ceph/mgr/mgr_module.py\", line 475, in call\n    return self.func(mgr, **kwargs)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 119, in <lambda>\n    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 108, in wrapper\n    return func(*args, **kwargs)\n  File \"/usr/share/ceph/mgr/orchestrator/module.py\", line 1306, in _daemon_add_osd\n    raise_if_exception(completion)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 240, in raise_if_exception\n    raise e\nRuntimeError: cephadm exited with an error code: 1, stderr:Inferring config /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config\nNon-zero exit code 1 from /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpd1rlzth8:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmp3f_afm7q:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdb --yes --no-systemd\n/usr/bin/docker: stderr  stderr: lsblk: /dev/vdb: not a block device\n/usr/bin/docker: stderr Traceback (most recent call last):\n/usr/bin/docker: stderr   File \"/usr/sbin/ceph-volume\", line 33, in <module>\n/usr/bin/docker: stderr     sys.exit(load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')())\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 54, in __init__\n/usr/bin/docker: stderr     self.main(self.argv)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/decorators.py\", line 59, in newfunc\n/usr/bin/docker: stderr     return f(*a, **kw)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 166, in main\n/usr/bin/docker: stderr     terminal.dispatch(self.mapper, subcommand_args)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 194, in dispatch\n/usr/bin/docker: stderr     instance.main()\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/main.py\", line 46, in main\n/usr/bin/docker: stderr     terminal.dispatch(self.mapper, self.argv)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 192, in dispatch\n/usr/bin/docker: stderr     instance = mapper.get(arg)(argv[count:])\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/batch.py\", line 325, in __init__\n/usr/bin/docker: stderr     self.args = parser.parse_args(argv)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1825, in parse_args\n/usr/bin/docker: stderr     args, argv = self.parse_known_args(args, namespace)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1858, in parse_known_args\n/usr/bin/docker: stderr     namespace, args = self._parse_known_args(args, namespace)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2049, in _parse_known_args\n/usr/bin/docker: stderr     positionals_end_index = consume_positionals(start_index)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2026, in consume_positionals\n/usr/bin/docker: stderr     take_action(action, args)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1919, in take_action\n/usr/bin/docker: stderr     argument_values = self._get_values(action, argument_strings)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in _get_values\n/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in <listcomp>\n/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2483, in _get_value\n/usr/bin/docker: stderr     result = type_func(arg_string)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 124, in __call__\n/usr/bin/docker: stderr     super().get_device(dev_path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 32, in get_device\n/usr/bin/docker: stderr     self._device = Device(dev_path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 140, in __init__\n/usr/bin/docker: stderr     self._parse()\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 236, in _parse\n/usr/bin/docker: stderr     dev = disk.lsblk(self.path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 244, in lsblk\n/usr/bin/docker: stderr     result = lsblk_all(device=device,\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 338, in lsblk_all\n/usr/bin/docker: stderr     raise RuntimeError(f\"Error: {err}\")\n/usr/bin/docker: stderr RuntimeError: Error: ['lsblk: /dev/vdb: not a block device']\nTraceback (most recent call last):\n  File \"/usr/lib/python3.10/runpy.py\", line 196, in _run_module_as_main\n    return _run_code(code, main_globals, None,\n  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code\n    exec(code, run_globals)\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 11009, in <module>\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 10997, in main\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2593, in _infer_config\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2509, in _infer_fsid\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2621, in _infer_image\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2496, in _validate_fsid\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 7226, in command_ceph_volume\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2284, in call_throws\nRuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpd1rlzth8:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmp3f_afm7q:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdb --yes --no-systemd", "stderr_lines": ["Error EINVAL: Traceback (most recent call last):", "  File \"/usr/share/ceph/mgr/mgr_module.py\", line 1834, in _handle_command", "    return self.handle_command(inbuf, cmd)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 183, in handle_command", "    return dispatch[cmd['prefix']].call(self, cmd, inbuf)", "  File \"/usr/share/ceph/mgr/mgr_module.py\", line 475, in call", "    return self.func(mgr, **kwargs)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 119, in <lambda>", "    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 108, in wrapper", "    return func(*args, **kwargs)", "  File \"/usr/share/ceph/mgr/orchestrator/module.py\", line 1306, in _daemon_add_osd", "    raise_if_exception(completion)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 240, in raise_if_exception", "    raise e", "RuntimeError: cephadm exited with an error code: 1, stderr:Inferring config /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config", "Non-zero exit code 1 from /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpd1rlzth8:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmp3f_afm7q:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdb --yes --no-systemd", "/usr/bin/docker: stderr  stderr: lsblk: /dev/vdb: not a block device", "/usr/bin/docker: stderr Traceback (most recent call last):", "/usr/bin/docker: stderr   File \"/usr/sbin/ceph-volume\", line 33, in <module>", "/usr/bin/docker: stderr     sys.exit(load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')())", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 54, in __init__", "/usr/bin/docker: stderr     self.main(self.argv)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/decorators.py\", line 59, in newfunc", "/usr/bin/docker: stderr     return f(*a, **kw)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 166, in main", "/usr/bin/docker: stderr     terminal.dispatch(self.mapper, subcommand_args)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 194, in dispatch", "/usr/bin/docker: stderr     instance.main()", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/main.py\", line 46, in main", "/usr/bin/docker: stderr     terminal.dispatch(self.mapper, self.argv)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 192, in dispatch", "/usr/bin/docker: stderr     instance = mapper.get(arg)(argv[count:])", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/batch.py\", line 325, in __init__", "/usr/bin/docker: stderr     self.args = parser.parse_args(argv)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1825, in parse_args", "/usr/bin/docker: stderr     args, argv = self.parse_known_args(args, namespace)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1858, in parse_known_args", "/usr/bin/docker: stderr     namespace, args = self._parse_known_args(args, namespace)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2049, in _parse_known_args", "/usr/bin/docker: stderr     positionals_end_index = consume_positionals(start_index)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2026, in consume_positionals", "/usr/bin/docker: stderr     take_action(action, args)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1919, in take_action", "/usr/bin/docker: stderr     argument_values = self._get_values(action, argument_strings)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in _get_values", "/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in <listcomp>", "/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2483, in _get_value", "/usr/bin/docker: stderr     result = type_func(arg_string)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 124, in __call__", "/usr/bin/docker: stderr     super().get_device(dev_path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 32, in get_device", "/usr/bin/docker: stderr     self._device = Device(dev_path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 140, in __init__", "/usr/bin/docker: stderr     self._parse()", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 236, in _parse", "/usr/bin/docker: stderr     dev = disk.lsblk(self.path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 244, in lsblk", "/usr/bin/docker: stderr     result = lsblk_all(device=device,", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 338, in lsblk_all", "/usr/bin/docker: stderr     raise RuntimeError(f\"Error: {err}\")", "/usr/bin/docker: stderr RuntimeError: Error: ['lsblk: /dev/vdb: not a block device']", "Traceback (most recent call last):", "  File \"/usr/lib/python3.10/runpy.py\", line 196, in _run_module_as_main", "    return _run_code(code, main_globals, None,", "  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code", "    exec(code, run_globals)", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 11009, in <module>", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 10997, in main", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2593, in _infer_config", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2509, in _infer_fsid", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2621, in _infer_image", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2496, in _validate_fsid", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 7226, in command_ceph_volume", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2284, in call_throws", "RuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpd1rlzth8:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmp3f_afm7q:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdb --yes --no-systemd"], "stdout": "", "stdout_lines": []}
2026-04-08 04:15:01.529190 | instance | failed: [instance] (item=/dev/vdc) => {"ansible_loop_var": "item", "changed": false, "cmd": ["cephadm", "shell", "--fsid", "4837cbf8-4f90-4300-b3f6-726c9b9f89b4", "--config", "/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config", "--", "ceph", "orch", "daemon", "add", "osd", "instance:/dev/vdc"], "delta": "0:00:07.533158", "end": "2026-04-08 04:15:01.499893", "item": "/dev/vdc", "msg": "non-zero return code", "rc": 22, "start": "2026-04-08 04:14:53.966735", "stderr": "Error EINVAL: Traceback (most recent call last):\n  File \"/usr/share/ceph/mgr/mgr_module.py\", line 1834, in _handle_command\n    return self.handle_command(inbuf, cmd)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 183, in handle_command\n    return dispatch[cmd['prefix']].call(self, cmd, inbuf)\n  File \"/usr/share/ceph/mgr/mgr_module.py\", line 475, in call\n    return self.func(mgr, **kwargs)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 119, in <lambda>\n    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 108, in wrapper\n    return func(*args, **kwargs)\n  File \"/usr/share/ceph/mgr/orchestrator/module.py\", line 1306, in _daemon_add_osd\n    raise_if_exception(completion)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 240, in raise_if_exception\n    raise e\nRuntimeError: cephadm exited with an error code: 1, stderr:Inferring config /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config\nNon-zero exit code 1 from /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpdzeh_3q0:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmppkxvsffi:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdc --yes --no-systemd\n/usr/bin/docker: stderr  stderr: lsblk: /dev/vdc: not a block device\n/usr/bin/docker: stderr Traceback (most recent call last):\n/usr/bin/docker: stderr   File \"/usr/sbin/ceph-volume\", line 33, in <module>\n/usr/bin/docker: stderr     sys.exit(load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')())\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 54, in __init__\n/usr/bin/docker: stderr     self.main(self.argv)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/decorators.py\", line 59, in newfunc\n/usr/bin/docker: stderr     return f(*a, **kw)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 166, in main\n/usr/bin/docker: stderr     terminal.dispatch(self.mapper, subcommand_args)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 194, in dispatch\n/usr/bin/docker: stderr     instance.main()\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/main.py\", line 46, in main\n/usr/bin/docker: stderr     terminal.dispatch(self.mapper, self.argv)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 192, in dispatch\n/usr/bin/docker: stderr     instance = mapper.get(arg)(argv[count:])\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/batch.py\", line 325, in __init__\n/usr/bin/docker: stderr     self.args = parser.parse_args(argv)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1825, in parse_args\n/usr/bin/docker: stderr     args, argv = self.parse_known_args(args, namespace)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1858, in parse_known_args\n/usr/bin/docker: stderr     namespace, args = self._parse_known_args(args, namespace)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2049, in _parse_known_args\n/usr/bin/docker: stderr     positionals_end_index = consume_positionals(start_index)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2026, in consume_positionals\n/usr/bin/docker: stderr     take_action(action, args)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1919, in take_action\n/usr/bin/docker: stderr     argument_values = self._get_values(action, argument_strings)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in _get_values\n/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in <listcomp>\n/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2483, in _get_value\n/usr/bin/docker: stderr     result = type_func(arg_string)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 124, in __call__\n/usr/bin/docker: stderr     super().get_device(dev_path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 32, in get_device\n/usr/bin/docker: stderr     self._device = Device(dev_path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 140, in __init__\n/usr/bin/docker: stderr     self._parse()\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 236, in _parse\n/usr/bin/docker: stderr     dev = disk.lsblk(self.path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 244, in lsblk\n/usr/bin/docker: stderr     result = lsblk_all(device=device,\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 338, in lsblk_all\n/usr/bin/docker: stderr     raise RuntimeError(f\"Error: {err}\")\n/usr/bin/docker: stderr RuntimeError: Error: ['lsblk: /dev/vdc: not a block device']\nTraceback (most recent call last):\n  File \"/usr/lib/python3.10/runpy.py\", line 196, in _run_module_as_main\n    return _run_code(code, main_globals, None,\n  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code\n    exec(code, run_globals)\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 11009, in <module>\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 10997, in main\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2593, in _infer_config\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2509, in _infer_fsid\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2621, in _infer_image\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2496, in _validate_fsid\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 7226, in command_ceph_volume\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2284, in call_throws\nRuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpdzeh_3q0:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmppkxvsffi:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdc --yes --no-systemd", "stderr_lines": ["Error EINVAL: Traceback (most recent call last):", "  File \"/usr/share/ceph/mgr/mgr_module.py\", line 1834, in _handle_command", "    return self.handle_command(inbuf, cmd)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 183, in handle_command", "    return dispatch[cmd['prefix']].call(self, cmd, inbuf)", "  File \"/usr/share/ceph/mgr/mgr_module.py\", line 475, in call", "    return self.func(mgr, **kwargs)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 119, in <lambda>", "    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 108, in wrapper", "    return func(*args, **kwargs)", "  File \"/usr/share/ceph/mgr/orchestrator/module.py\", line 1306, in _daemon_add_osd", "    raise_if_exception(completion)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 240, in raise_if_exception", "    raise e", "RuntimeError: cephadm exited with an error code: 1, stderr:Inferring config /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config", "Non-zero exit code 1 from /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpdzeh_3q0:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmppkxvsffi:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdc --yes --no-systemd", "/usr/bin/docker: stderr  stderr: lsblk: /dev/vdc: not a block device", "/usr/bin/docker: stderr Traceback (most recent call last):", "/usr/bin/docker: stderr   File \"/usr/sbin/ceph-volume\", line 33, in <module>", "/usr/bin/docker: stderr     sys.exit(load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')())", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 54, in __init__", "/usr/bin/docker: stderr     self.main(self.argv)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/decorators.py\", line 59, in newfunc", "/usr/bin/docker: stderr     return f(*a, **kw)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 166, in main", "/usr/bin/docker: stderr     terminal.dispatch(self.mapper, subcommand_args)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 194, in dispatch", "/usr/bin/docker: stderr     instance.main()", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/main.py\", line 46, in main", "/usr/bin/docker: stderr     terminal.dispatch(self.mapper, self.argv)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 192, in dispatch", "/usr/bin/docker: stderr     instance = mapper.get(arg)(argv[count:])", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/batch.py\", line 325, in __init__", "/usr/bin/docker: stderr     self.args = parser.parse_args(argv)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1825, in parse_args", "/usr/bin/docker: stderr     args, argv = self.parse_known_args(args, namespace)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1858, in parse_known_args", "/usr/bin/docker: stderr     namespace, args = self._parse_known_args(args, namespace)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2049, in _parse_known_args", "/usr/bin/docker: stderr     positionals_end_index = consume_positionals(start_index)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2026, in consume_positionals", "/usr/bin/docker: stderr     take_action(action, args)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1919, in take_action", "/usr/bin/docker: stderr     argument_values = self._get_values(action, argument_strings)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in _get_values", "/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in <listcomp>", "/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2483, in _get_value", "/usr/bin/docker: stderr     result = type_func(arg_string)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 124, in __call__", "/usr/bin/docker: stderr     super().get_device(dev_path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 32, in get_device", "/usr/bin/docker: stderr     self._device = Device(dev_path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 140, in __init__", "/usr/bin/docker: stderr     self._parse()", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 236, in _parse", "/usr/bin/docker: stderr     dev = disk.lsblk(self.path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 244, in lsblk", "/usr/bin/docker: stderr     result = lsblk_all(device=device,", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 338, in lsblk_all", "/usr/bin/docker: stderr     raise RuntimeError(f\"Error: {err}\")", "/usr/bin/docker: stderr RuntimeError: Error: ['lsblk: /dev/vdc: not a block device']", "Traceback (most recent call last):", "  File \"/usr/lib/python3.10/runpy.py\", line 196, in _run_module_as_main", "    return _run_code(code, main_globals, None,", "  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code", "    exec(code, run_globals)", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 11009, in <module>", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 10997, in main", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2593, in _infer_config", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2509, in _infer_fsid", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2621, in _infer_image", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2496, in _validate_fsid", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 7226, in command_ceph_volume", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2284, in call_throws", "RuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpdzeh_3q0:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmppkxvsffi:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdc --yes --no-systemd"], "stdout": "", "stdout_lines": []}
2026-04-08 04:15:09.166767 | instance | failed: [instance] (item=/dev/vdd) => {"ansible_loop_var": "item", "changed": false, "cmd": ["cephadm", "shell", "--fsid", "4837cbf8-4f90-4300-b3f6-726c9b9f89b4", "--config", "/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config", "--", "ceph", "orch", "daemon", "add", "osd", "instance:/dev/vdd"], "delta": "0:00:07.464273", "end": "2026-04-08 04:15:09.137898", "item": "/dev/vdd", "msg": "non-zero return code", "rc": 22, "start": "2026-04-08 04:15:01.673625", "stderr": "Error EINVAL: Traceback (most recent call last):\n  File \"/usr/share/ceph/mgr/mgr_module.py\", line 1834, in _handle_command\n    return self.handle_command(inbuf, cmd)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 183, in handle_command\n    return dispatch[cmd['prefix']].call(self, cmd, inbuf)\n  File \"/usr/share/ceph/mgr/mgr_module.py\", line 475, in call\n    return self.func(mgr, **kwargs)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 119, in <lambda>\n    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 108, in wrapper\n    return func(*args, **kwargs)\n  File \"/usr/share/ceph/mgr/orchestrator/module.py\", line 1306, in _daemon_add_osd\n    raise_if_exception(completion)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 240, in raise_if_exception\n    raise e\nRuntimeError: cephadm exited with an error code: 1, stderr:Inferring config /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config\nNon-zero exit code 1 from /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpp6hv_74y:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmpuh78a2c0:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdd --yes --no-systemd\n/usr/bin/docker: stderr  stderr: lsblk: /dev/vdd: not a block device\n/usr/bin/docker: stderr Traceback (most recent call last):\n/usr/bin/docker: stderr   File \"/usr/sbin/ceph-volume\", line 33, in <module>\n/usr/bin/docker: stderr     sys.exit(load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')())\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 54, in __init__\n/usr/bin/docker: stderr     self.main(self.argv)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/decorators.py\", line 59, in newfunc\n/usr/bin/docker: stderr     return f(*a, **kw)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 166, in main\n/usr/bin/docker: stderr     terminal.dispatch(self.mapper, subcommand_args)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 194, in dispatch\n/usr/bin/docker: stderr     instance.main()\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/main.py\", line 46, in main\n/usr/bin/docker: stderr     terminal.dispatch(self.mapper, self.argv)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 192, in dispatch\n/usr/bin/docker: stderr     instance = mapper.get(arg)(argv[count:])\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/batch.py\", line 325, in __init__\n/usr/bin/docker: stderr     self.args = parser.parse_args(argv)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1825, in parse_args\n/usr/bin/docker: stderr     args, argv = self.parse_known_args(args, namespace)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1858, in parse_known_args\n/usr/bin/docker: stderr     namespace, args = self._parse_known_args(args, namespace)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2049, in _parse_known_args\n/usr/bin/docker: stderr     positionals_end_index = consume_positionals(start_index)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2026, in consume_positionals\n/usr/bin/docker: stderr     take_action(action, args)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1919, in take_action\n/usr/bin/docker: stderr     argument_values = self._get_values(action, argument_strings)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in _get_values\n/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in <listcomp>\n/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2483, in _get_value\n/usr/bin/docker: stderr     result = type_func(arg_string)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 124, in __call__\n/usr/bin/docker: stderr     super().get_device(dev_path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 32, in get_device\n/usr/bin/docker: stderr     self._device = Device(dev_path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 140, in __init__\n/usr/bin/docker: stderr     self._parse()\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 236, in _parse\n/usr/bin/docker: stderr     dev = disk.lsblk(self.path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 244, in lsblk\n/usr/bin/docker: stderr     result = lsblk_all(device=device,\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 338, in lsblk_all\n/usr/bin/docker: stderr     raise RuntimeError(f\"Error: {err}\")\n/usr/bin/docker: stderr RuntimeError: Error: ['lsblk: /dev/vdd: not a block device']\nTraceback (most recent call last):\n  File \"/usr/lib/python3.10/runpy.py\", line 196, in _run_module_as_main\n    return _run_code(code, main_globals, None,\n  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code\n    exec(code, run_globals)\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 11009, in <module>\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 10997, in main\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2593, in _infer_config\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2509, in _infer_fsid\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2621, in _infer_image\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2496, in _validate_fsid\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 7226, in command_ceph_volume\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2284, in call_throws\nRuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpp6hv_74y:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmpuh78a2c0:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdd --yes --no-systemd", "stderr_lines": ["Error EINVAL: Traceback (most recent call last):", "  File \"/usr/share/ceph/mgr/mgr_module.py\", line 1834, in _handle_command", "    return self.handle_command(inbuf, cmd)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 183, in handle_command", "    return dispatch[cmd['prefix']].call(self, cmd, inbuf)", "  File \"/usr/share/ceph/mgr/mgr_module.py\", line 475, in call", "    return self.func(mgr, **kwargs)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 119, in <lambda>", "    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 108, in wrapper", "    return func(*args, **kwargs)", "  File \"/usr/share/ceph/mgr/orchestrator/module.py\", line 1306, in _daemon_add_osd", "    raise_if_exception(completion)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 240, in raise_if_exception", "    raise e", "RuntimeError: cephadm exited with an error code: 1, stderr:Inferring config /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config", "Non-zero exit code 1 from /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpp6hv_74y:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmpuh78a2c0:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdd --yes --no-systemd", "/usr/bin/docker: stderr  stderr: lsblk: /dev/vdd: not a block device", "/usr/bin/docker: stderr Traceback (most recent call last):", "/usr/bin/docker: stderr   File \"/usr/sbin/ceph-volume\", line 33, in <module>", "/usr/bin/docker: stderr     sys.exit(load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')())", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 54, in __init__", "/usr/bin/docker: stderr     self.main(self.argv)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/decorators.py\", line 59, in newfunc", "/usr/bin/docker: stderr     return f(*a, **kw)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 166, in main", "/usr/bin/docker: stderr     terminal.dispatch(self.mapper, subcommand_args)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 194, in dispatch", "/usr/bin/docker: stderr     instance.main()", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/main.py\", line 46, in main", "/usr/bin/docker: stderr     terminal.dispatch(self.mapper, self.argv)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 192, in dispatch", "/usr/bin/docker: stderr     instance = mapper.get(arg)(argv[count:])", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/batch.py\", line 325, in __init__", "/usr/bin/docker: stderr     self.args = parser.parse_args(argv)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1825, in parse_args", "/usr/bin/docker: stderr     args, argv = self.parse_known_args(args, namespace)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1858, in parse_known_args", "/usr/bin/docker: stderr     namespace, args = self._parse_known_args(args, namespace)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2049, in _parse_known_args", "/usr/bin/docker: stderr     positionals_end_index = consume_positionals(start_index)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2026, in consume_positionals", "/usr/bin/docker: stderr     take_action(action, args)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1919, in take_action", "/usr/bin/docker: stderr     argument_values = self._get_values(action, argument_strings)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in _get_values", "/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in <listcomp>", "/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2483, in _get_value", "/usr/bin/docker: stderr     result = type_func(arg_string)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 124, in __call__", "/usr/bin/docker: stderr     super().get_device(dev_path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 32, in get_device", "/usr/bin/docker: stderr     self._device = Device(dev_path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 140, in __init__", "/usr/bin/docker: stderr     self._parse()", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 236, in _parse", "/usr/bin/docker: stderr     dev = disk.lsblk(self.path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 244, in lsblk", "/usr/bin/docker: stderr     result = lsblk_all(device=device,", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 338, in lsblk_all", "/usr/bin/docker: stderr     raise RuntimeError(f\"Error: {err}\")", "/usr/bin/docker: stderr RuntimeError: Error: ['lsblk: /dev/vdd: not a block device']", "Traceback (most recent call last):", "  File \"/usr/lib/python3.10/runpy.py\", line 196, in _run_module_as_main", "    return _run_code(code, main_globals, None,", "  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code", "    exec(code, run_globals)", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 11009, in <module>", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 10997, in main", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2593, in _infer_config", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2509, in _infer_fsid", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2621, in _infer_image", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2496, in _validate_fsid", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 7226, in command_ceph_volume", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2284, in call_throws", "RuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpp6hv_74y:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmpuh78a2c0:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdd --yes --no-systemd"], "stdout": "", "stdout_lines": []}
2026-04-08 04:15:09.172135 | instance |
2026-04-08 04:15:09.175035 | instance | PLAY RECAP *********************************************************************
2026-04-08 04:15:09.175096 | instance | instance                   : ok=105  changed=26   unreachable=0    failed=1    skipped=26   rescued=0    ignored=0
2026-04-08 04:15:09.175108 | instance |
2026-04-08 04:15:09.175118 | instance | Wednesday 08 April 2026  04:15:09 +0000 (0:00:22.989)       0:04:12.002 *******
2026-04-08 04:15:09.175127 | instance | ===============================================================================
2026-04-08 04:15:09.175136 | instance | vexxhost.ceph.mon : Run Bootstrap coomand ----------------------------- 124.92s
2026-04-08 04:15:09.175145 | instance | vexxhost.ceph.osd : Install OSDs --------------------------------------- 22.99s
2026-04-08 04:15:09.175153 | instance | vexxhost.ceph.osd : Get `ceph-volume lvm list` status ------------------ 10.39s
2026-04-08 04:15:09.175162 | instance | vexxhost.ceph.mon : Validate monitor exist ----------------------------- 10.18s
2026-04-08 04:15:09.175170 | instance | vexxhost.ceph.osd : Ensure all OSDs are non-legacy ---------------------- 5.34s
2026-04-08 04:15:09.175179 | instance | vexxhost.ceph.osd : Get `cephadm ls` status ----------------------------- 5.27s
2026-04-08 04:15:09.175187 | instance | vexxhost.containers.containerd : Install AppArmor packages -------------- 5.17s
2026-04-08 04:15:09.175196 | instance | vexxhost.ceph.cephadm : Install packages -------------------------------- 4.72s
2026-04-08 04:15:09.175204 | instance | vexxhost.containers.download_artifact : Extract archive ----------------- 4.20s
2026-04-08 04:15:09.175213 | instance | vexxhost.containers.download_artifact : Extract archive ----------------- 3.05s
2026-04-08 04:15:09.175221 | instance | vexxhost.containers.download_artifact : Extract archive ----------------- 2.82s
2026-04-08 04:15:09.175237 | instance | vexxhost.ceph.mgr : Enable the Ceph Manager prometheus module ----------- 2.11s
2026-04-08 04:15:09.175311 | instance | vexxhost.ceph.cephadm_host : Add new host to Ceph ----------------------- 1.96s
2026-04-08 04:15:09.175552 | instance | vexxhost.ceph.cephadm_host : Add new host to Ceph ----------------------- 1.93s
2026-04-08 04:15:09.175763 | instance | vexxhost.containers.download_artifact : Extract archive ----------------- 1.90s
2026-04-08 04:15:09.175972 | instance | vexxhost.ceph.cephadm_host : Add new host to Ceph ----------------------- 1.87s
2026-04-08 04:15:09.176181 | instance | vexxhost.containers.containerd : Reload systemd ------------------------- 1.69s
2026-04-08 04:15:09.176392 | instance | vexxhost.ceph.mon : Get `cephadm ls` status ----------------------------- 1.62s
2026-04-08 04:15:09.176623 | instance | vexxhost.ceph.mgr : Configure "mgr" label for managers ------------------ 1.60s
2026-04-08 04:15:09.176839 | instance | vexxhost.ceph.mon : Configure "mon" label for monitors ------------------ 1.59s
2026-04-08 04:15:09.333605 | instance | CRITICAL Ansible return code was 2, command was: ansible-playbook --inventory /home/zuul/.ansible/tmp/molecule.v9Wo.aio/inventory --skip-tags molecule-notest,notest --inventory=/home/zuul/src/github.com/vexxhost/atmosphere/inventory.yaml /home/zuul/src/github.com/vexxhost/atmosphere/molecule/aio/converge.yml
2026-04-08 04:15:09.333693 | instance | ERROR    [aio > converge] Executed: Failed
2026-04-08 04:15:09.333762 | instance | ERROR    Ansible return code was 2, command was: ansible-playbook --inventory /home/zuul/.ansible/tmp/molecule.v9Wo.aio/inventory --skip-tags molecule-notest,notest --inventory=/home/zuul/src/github.com/vexxhost/atmosphere/inventory.yaml /home/zuul/src/github.com/vexxhost/atmosphere/molecule/aio/converge.yml
2026-04-08 04:15:09.814642 | instance | ERROR
2026-04-08 04:15:09.814885 | instance | {
2026-04-08 04:15:09.814925 | instance |   "delta": "0:06:24.337421",
2026-04-08 04:15:09.814954 | instance |   "end": "2026-04-08 04:15:09.412626",
2026-04-08 04:15:09.815001 | instance |   "msg": "non-zero return code",
2026-04-08 04:15:09.815028 | instance |   "rc": 2,
2026-04-08 04:15:09.815057 | instance |   "start": "2026-04-08 04:08:45.075205"
2026-04-08 04:15:09.815085 | instance | }
failure
2026-04-08 04:15:09.831933 | 
2026-04-08 04:15:09.832122 | PLAY RECAP
2026-04-08 04:15:09.832232 | instance | ok: 2 changed: 2 unreachable: 0 failed: 1 skipped: 0 rescued: 0 ignored: 0
2026-04-08 04:15:09.832291 | 
2026-04-08 04:15:09.978660 | RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/run.yaml@main]
2026-04-08 04:15:09.988931 | POST-RUN START: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/post.yml@main]
2026-04-08 04:15:10.618402 | 
2026-04-08 04:15:10.618544 | PLAY [all]
2026-04-08 04:15:10.633145 | 
2026-04-08 04:15:10.633228 | TASK [gather-host-logs : creating directory for system status]
2026-04-08 04:15:10.970208 | instance | changed
2026-04-08 04:15:10.976135 | 
2026-04-08 04:15:10.976214 | TASK [gather-host-logs : Get logs for each host]
2026-04-08 04:15:11.308242 | instance | + systemd-cgls --full --all --no-pager
2026-04-08 04:15:11.323559 | instance | + ip addr
2026-04-08 04:15:11.325971 | instance | + ip route
2026-04-08 04:15:11.328522 | instance | + lsblk
2026-04-08 04:15:11.331830 | instance | + mount
2026-04-08 04:15:11.334356 | instance | + docker images
2026-04-08 04:15:11.351667 | instance | + brctl show
2026-04-08 04:15:11.352079 | instance | /bin/bash: line 8: brctl: command not found
2026-04-08 04:15:11.352242 | instance | + ps aux --sort=-%mem
2026-04-08 04:15:11.381938 | instance | + dpkg -l
2026-04-08 04:15:11.392306 | instance | + CONTAINERS=($(docker ps -a --format '{{ .Names }}' --filter label=zuul))
2026-04-08 04:15:11.392913 | instance | ++ docker ps -a --format '{{ .Names }}' --filter label=zuul
2026-04-08 04:15:11.410267 | instance | + '[' '!' -z '' ']'
2026-04-08 04:15:11.514779 | instance | ok: Runtime: 0:00:00.106610
2026-04-08 04:15:11.521514 | 
2026-04-08 04:15:11.521582 | TASK [gather-host-logs : Downloads logs to executor]
2026-04-08 04:15:12.143439 | instance | changed:
2026-04-08 04:15:12.143736 | instance | created directory /var/lib/zuul/builds/aed307efefdd4e9b8baf020d172c9ed9/work/logs/instance
2026-04-08 04:15:12.143804 | instance | cd+++++++++ system/
2026-04-08 04:15:12.143864 | instance | >f+++++++++ system/brctl-show.txt
2026-04-08 04:15:12.143912 | instance | >f+++++++++ system/docker-images.txt
2026-04-08 04:15:12.143936 | instance | >f+++++++++ system/ip-addr.txt
2026-04-08 04:15:12.143962 | instance | >f+++++++++ system/ip-route.txt
2026-04-08 04:15:12.143983 | instance | >f+++++++++ system/lsblk.txt
2026-04-08 04:15:12.144003 | instance | >f+++++++++ system/mount.txt
2026-04-08 04:15:12.144022 | instance | >f+++++++++ system/packages.txt
2026-04-08 04:15:12.144041 | instance | >f+++++++++ system/ps.txt
2026-04-08 04:15:12.144061 | instance | >f+++++++++ system/systemd-cgls.txt
2026-04-08 04:15:12.153168 | 
2026-04-08 04:15:12.153234 | LOOP [helm-release-status : creating directory for helm release status]
2026-04-08 04:15:12.352622 | instance | changed: "values"
2026-04-08 04:15:12.515554 | instance | changed: "releases"
2026-04-08 04:15:12.525162 | 
2026-04-08 04:15:12.525322 | TASK [helm-release-status : Gather get release status for helm charts]
2026-04-08 04:15:12.738662 | instance | /bin/bash: line 3: kubectl: command not found
2026-04-08 04:15:13.064794 | instance | ok: Runtime: 0:00:00.005451
2026-04-08 04:15:13.071516 | 
2026-04-08 04:15:13.071582 | TASK [helm-release-status : Downloads logs to executor]
2026-04-08 04:15:13.572934 | instance | changed:
2026-04-08 04:15:13.573140 | instance | cd+++++++++ helm/
2026-04-08 04:15:13.573167 | instance | cd+++++++++ helm/releases/
2026-04-08 04:15:13.573188 | instance | cd+++++++++ helm/values/
2026-04-08 04:15:13.582203 | 
2026-04-08 04:15:13.582274 | TASK [describe-kubernetes-objects : creating directory for cluster scoped objects]
2026-04-08 04:15:13.779845 | instance | changed
2026-04-08 04:15:13.816768 | 
2026-04-08 04:15:13.816921 | TASK [describe-kubernetes-objects : Gathering descriptions for cluster scoped objects]
2026-04-08 04:15:14.013933 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
2026-04-08 04:15:14.014028 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
2026-04-08 04:15:14.019958 | instance | environment: line 1: kubectl: command not found
2026-04-08 04:15:14.021302 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value
2026-04-08 04:15:14.021694 | instance | environment: line 1: kubectl: command not found
2026-04-08 04:15:14.023246 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value
2026-04-08 04:15:14.024960 | instance | environment: line 1: kubectl: command not found
2026-04-08 04:15:14.026223 | instance | environment: line 1: kubectl: command not found
2026-04-08 04:15:14.026683 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value
2026-04-08 04:15:14.028305 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value
2026-04-08 04:15:14.029863 | instance | environment: line 1: kubectl: command not found
2026-04-08 04:15:14.031176 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value
2026-04-08 04:15:14.354713 | instance | ok: Runtime: 0:00:00.024637
2026-04-08 04:15:14.360598 | 
2026-04-08 04:15:14.360658 | TASK [describe-kubernetes-objects : creating directory for namespace scoped objects]
2026-04-08 04:15:14.547394 | instance | changed
2026-04-08 04:15:14.555077 | 
2026-04-08 04:15:14.555213 | TASK [describe-kubernetes-objects : Gathering descriptions for namespace scoped objects]
2026-04-08 04:15:14.763719 | instance | environment: line 5: kubectl: command not found
2026-04-08 04:15:14.764336 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
2026-04-08 04:15:14.764646 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
2026-04-08 04:15:14.764927 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
2026-04-08 04:15:15.087274 | instance | ok: Runtime: 0:00:00.008162
2026-04-08 04:15:15.096159 | 
2026-04-08 04:15:15.096221 | TASK [describe-kubernetes-objects : Downloads logs to executor]
2026-04-08 04:15:15.584115 | instance | changed:
2026-04-08 04:15:15.584356 | instance | cd+++++++++ objects/
2026-04-08 04:15:15.584395 | instance | cd+++++++++ objects/cluster/
2026-04-08 04:15:15.584426 | instance | cd+++++++++ objects/namespaced/
2026-04-08 04:15:15.595836 | 
2026-04-08 04:15:15.595907 | TASK [gather-pod-logs : creating directory for pod logs]
2026-04-08 04:15:15.783644 | instance | changed
2026-04-08 04:15:15.790794 | 
2026-04-08 04:15:15.790868 | TASK [gather-pod-logs : creating directory for failed pod logs]
2026-04-08 04:15:15.978402 | instance | changed
2026-04-08 04:15:15.983716 | 
2026-04-08 04:15:15.983790 | TASK [gather-pod-logs : retrieve all kubernetes logs, current and previous (if they exist)]
2026-04-08 04:15:16.186081 | instance | environment: line 3: kubectl: command not found
2026-04-08 04:15:16.520389 | instance | ok: Runtime: 0:00:00.007398
2026-04-08 04:15:16.527200 | 
2026-04-08 04:15:16.527280 | TASK [gather-pod-logs : Downloads pod logs to executor]
2026-04-08 04:15:17.020365 | instance | changed:
2026-04-08 04:15:17.020503 | instance | cd+++++++++ pod-logs/
2026-04-08 04:15:17.020530 | instance | cd+++++++++ pod-logs/failed-pods/
2026-04-08 04:15:17.030031 | 
2026-04-08 04:15:17.030090 | TASK [gather-prom-metrics : creating directory for helm release descriptions]
2026-04-08 04:15:17.216370 | instance | changed
2026-04-08 04:15:17.221167 | 
2026-04-08 04:15:17.221234 | TASK [gather-prom-metrics : Get metrics from exporter services in all namespaces]
2026-04-08 04:15:17.421003 | instance | /bin/bash: line 2: kubectl: command not found
2026-04-08 04:15:17.757374 | instance | ok: Runtime: 0:00:00.033479
2026-04-08 04:15:17.764458 | 
2026-04-08 04:15:17.764547 | TASK [gather-prom-metrics : Get ceph metrics from ceph-mgr]
2026-04-08 04:15:17.964772 | instance | /bin/bash: line 2: kubectl: command not found
2026-04-08 04:15:17.992538 | instance | ceph-mgr endpoints:
2026-04-08 04:15:18.301146 | instance | ok: Runtime: 0:00:00.033449
2026-04-08 04:15:18.307806 | 
2026-04-08 04:15:18.307869 | TASK [gather-prom-metrics : Get metrics from fluentd pods]
2026-04-08 04:15:18.512540 | instance | /bin/bash: line 4: kubectl: command not found
2026-04-08 04:15:18.843216 | instance | ok: Runtime: 0:00:00.032850
2026-04-08 04:15:18.850738 | 
2026-04-08 04:15:18.850822 | TASK [gather-prom-metrics : Downloads logs to executor]
2026-04-08 04:15:19.344311 | instance | changed: cd+++++++++ prometheus/
2026-04-08 04:15:19.353321 | 
2026-04-08 04:15:19.353381 | TASK [gather-selenium-data : creating directory for helm release descriptions]
2026-04-08 04:15:19.588436 | instance | changed
2026-04-08 04:15:19.593837 | 
2026-04-08 04:15:19.593897 | TASK [gather-selenium-data : Get selenium data]
2026-04-08 04:15:19.792850 | instance | + cp '/tmp/artifacts/*' /tmp/logs/selenium/.
2026-04-08 04:15:19.794260 | instance | cp: cannot stat '/tmp/artifacts/*': No such file or directory
2026-04-08 04:15:20.129986 | instance | ERROR
2026-04-08 04:15:20.130309 | instance | {
2026-04-08 04:15:20.130381 | instance |   "delta": "0:00:00.005702",
2026-04-08 04:15:20.130429 | instance |   "end": "2026-04-08 04:15:19.794622",
2026-04-08 04:15:20.130477 | instance |   "msg": "non-zero return code",
2026-04-08 04:15:20.130520 | instance |   "rc": 1,
2026-04-08 04:15:20.130562 | instance |   "start": "2026-04-08 04:15:19.788920"
2026-04-08 04:15:20.130603 | instance | }
2026-04-08 04:15:20.130655 | instance | ERROR: Ignoring Errors
2026-04-08 04:15:20.137925 | 
2026-04-08 04:15:20.138012 | TASK [gather-selenium-data : Downloads logs to executor]
2026-04-08 04:15:20.619534 | instance | changed: cd+++++++++ selenium/
2026-04-08 04:15:20.627722 | 
2026-04-08 04:15:20.627788 | PLAY RECAP
2026-04-08 04:15:20.627851 | instance | ok: 23 changed: 23 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 1
2026-04-08 04:15:20.627882 | 
2026-04-08 04:15:20.740072 | POST-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/post.yml@main]
2026-04-08 04:15:20.759596 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main]
2026-04-08 04:15:21.348130 | 
2026-04-08 04:15:21.348607 | PLAY [all]
2026-04-08 04:15:21.359819 | 
2026-04-08 04:15:21.359890 | TASK [fetch-output : Set log path for multiple nodes]
2026-04-08 04:15:21.405465 | instance | skipping: Conditional result was False
2026-04-08 04:15:21.417224 | 
2026-04-08 04:15:21.417310 | TASK [fetch-output : Set log path for single node]
2026-04-08 04:15:21.461838 | instance | ok
2026-04-08 04:15:21.467627 | 
2026-04-08 04:15:21.467694 | LOOP [fetch-output : Ensure local output dirs]
2026-04-08 04:15:21.852705 | instance -> localhost | ok: "/var/lib/zuul/builds/aed307efefdd4e9b8baf020d172c9ed9/work/logs"
2026-04-08 04:15:22.078748 | instance -> localhost | changed: "/var/lib/zuul/builds/aed307efefdd4e9b8baf020d172c9ed9/work/artifacts"
2026-04-08 04:15:22.282510 | instance -> localhost | changed: "/var/lib/zuul/builds/aed307efefdd4e9b8baf020d172c9ed9/work/docs"
2026-04-08 04:15:22.302986 | 
2026-04-08 04:15:22.303128 | LOOP [fetch-output : Collect logs, artifacts and docs]
2026-04-08 04:15:22.936150 | instance | changed: .d..t...... ./
2026-04-08 04:15:22.936395 | instance | changed: All items complete
2026-04-08 04:15:22.936423 | 
2026-04-08 04:15:23.399036 | instance | changed: .d..t...... ./
2026-04-08 04:15:23.843330 | instance | changed: .d..t...... ./
2026-04-08 04:15:23.854534 | 
2026-04-08 04:15:23.854615 | LOOP [merge-output-to-logs : Move artifacts and docs to logs dir]
2026-04-08 04:15:24.271918 | instance -> localhost | ok: Item: artifacts Runtime: 0:00:00.007391
2026-04-08 04:15:24.492378 | instance -> localhost | ok: Item: docs Runtime: 0:00:00.008278
2026-04-08 04:15:24.499663 | 
2026-04-08 04:15:24.499719 | PLAY [all]
2026-04-08 04:15:24.506216 | 
2026-04-08 04:15:24.506276 | TASK [remove-build-sshkey : Remove the build SSH key from all nodes]
2026-04-08 04:15:24.915342 | instance | changed
2026-04-08 04:15:24.921257 | 
2026-04-08 04:15:24.921313 | PLAY RECAP
2026-04-08 04:15:24.921361 | instance | ok: 5 changed: 4 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0
2026-04-08 04:15:24.921382 | 
2026-04-08 04:15:25.040533 | POST-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main]
2026-04-08 04:15:25.052628 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post-logs.yaml@main]
2026-04-08 04:15:25.649929 | 
2026-04-08 04:15:25.650103 | PLAY [localhost]
2026-04-08 04:15:25.660700 | 
2026-04-08 04:15:25.660774 | TASK [Generate Zuul manifest]
2026-04-08 04:15:25.684934 | localhost | ok
2026-04-08 04:15:25.701509 | 
2026-04-08 04:15:25.701611 | TASK [generate-zuul-manifest : Generate Zuul manifest]
2026-04-08 04:15:26.080554 | localhost | changed
2026-04-08 04:15:26.092894 | 
2026-04-08 04:15:26.093006 | TASK [generate-zuul-manifest : Return Zuul manifest URL to Zuul]
2026-04-08 04:15:26.124864 | localhost | ok
2026-04-08 04:15:26.133053 | 
2026-04-08 04:15:26.133118 | TASK [Upload logs]
2026-04-08 04:15:26.153516 | localhost | ok
2026-04-08 04:15:26.269311 | 
2026-04-08 04:15:26.269479 | TASK [Set zuul-log-path fact]
2026-04-08 04:15:26.292471 | localhost | ok
2026-04-08 04:15:26.306310 | 
2026-04-08 04:15:26.306374 | TASK [set-zuul-log-path-fact : Set log path for a build]
2026-04-08 04:15:26.338333 | localhost | ok
2026-04-08 04:15:26.346698 | 
2026-04-08 04:15:26.346762 | TASK [upload-logs : Create log directories]
2026-04-08 04:15:26.714042 | localhost | changed
2026-04-08 04:15:26.720937 | 
2026-04-08 04:15:26.721065 | TASK [upload-logs : Ensure logs are readable before uploading]
2026-04-08 04:15:27.094748 | localhost -> localhost | ok: Runtime: 0:00:00.005469
2026-04-08 04:15:27.102017 | 
2026-04-08 04:15:27.102110 | TASK [upload-logs : Upload logs to log server]
2026-04-08 04:15:27.560912 | localhost | Output suppressed because no_log was given
2026-04-08 04:15:27.565764 | 
2026-04-08 04:15:27.565850 | LOOP [upload-logs : Compress console log and json output]
2026-04-08 04:15:27.611354 | localhost | skipping: Conditional result was False
2026-04-08 04:15:27.618465 | localhost | skipping: Conditional result was False
2026-04-08 04:15:27.628941 | 
2026-04-08 04:15:27.629117 | LOOP [upload-logs : Upload compressed console log and json output]
2026-04-08 04:15:27.674610 | localhost | skipping: Conditional result was False
2026-04-08 04:15:27.675070 | 
2026-04-08 04:15:27.678903 | localhost | skipping: Conditional result was False
2026-04-08 04:15:27.694848 | 
2026-04-08 04:15:27.695044 | LOOP [upload-logs : Upload console log and json output]