2026-04-08 03:05:47.938607 | Job console starting
2026-04-08 03:05:47.947101 | Updating git repos
2026-04-08 03:05:48.005248 | Cloning repos into workspace
2026-04-08 03:05:48.068371 | Restoring repo states
2026-04-08 03:05:48.088557 | Merging changes
2026-04-08 03:05:49.033704 | Checking out repos
2026-04-08 03:05:49.172799 | Preparing playbooks
2026-04-08 03:05:51.524830 | Running Ansible setup
2026-04-08 03:05:55.344306 | PRE-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main]
2026-04-08 03:05:55.943347 | 
2026-04-08 03:05:55.943449 | PLAY [localhost]
2026-04-08 03:05:55.950515 | 
2026-04-08 03:05:55.950583 | TASK [Gathering Facts]
2026-04-08 03:05:56.857639 | localhost | ok
2026-04-08 03:05:56.868625 | 
2026-04-08 03:05:56.868744 | TASK [Setup log path fact]
2026-04-08 03:05:56.889214 | localhost | ok
2026-04-08 03:05:56.903709 | 
2026-04-08 03:05:56.903793 | TASK [set-zuul-log-path-fact : Set log path for a build]
2026-04-08 03:05:56.933333 | localhost | ok
2026-04-08 03:05:56.941064 | 
2026-04-08 03:05:56.941153 | TASK [emit-job-header : Print job information]
2026-04-08 03:05:56.980375 | # Job Information
2026-04-08 03:05:56.980531 | Ansible Version: 2.16.16
2026-04-08 03:05:56.980575 | Job: atmosphere-molecule-aio-openvswitch
2026-04-08 03:05:56.980604 | Pipeline: check
2026-04-08 03:05:56.980632 | Executor: 0a8996d2b663
2026-04-08 03:05:56.980659 | Triggered by: https://github.com/vexxhost/atmosphere/pull/3809
2026-04-08 03:05:56.980690 | Event ID: c20822a0-32f7-11f1-82a7-da0ae58ed532
2026-04-08 03:05:56.984644 | 
2026-04-08 03:05:56.984728 | LOOP [emit-job-header : Print node information]
2026-04-08 03:05:57.072982 | localhost | ok:
2026-04-08 03:05:57.073171 | localhost | # Node Information
2026-04-08 03:05:57.073209 | localhost | Inventory Hostname: instance
2026-04-08 03:05:57.073238 | localhost | Hostname: np0000164178
2026-04-08 03:05:57.073559 | localhost | Username: zuul
2026-04-08 03:05:57.073614 | localhost | Distro: Ubuntu 22.04
2026-04-08 03:05:57.073645 | localhost | Provider: yul1
2026-04-08 03:05:57.073672 | localhost | Region: ca-ymq-1
2026-04-08 03:05:57.073714 | localhost | Label: ubuntu-jammy-16
2026-04-08 03:05:57.073743 | localhost | Product Name: OpenStack Nova
2026-04-08 03:05:57.073769 | localhost | Interface IP: 162.253.55.204
2026-04-08 03:05:57.094503 | 
2026-04-08 03:05:57.094641 | TASK [log-inventory : Ensure Zuul Ansible directory exists]
2026-04-08 03:05:57.532642 | localhost -> localhost | changed
2026-04-08 03:05:57.540249 | 
2026-04-08 03:05:57.540340 | TASK [log-inventory : Copy ansible inventory to logs dir]
2026-04-08 03:05:58.518983 | localhost -> localhost | changed
2026-04-08 03:05:58.526000 | 
2026-04-08 03:05:58.526056 | PLAY [all]
2026-04-08 03:05:58.533590 | 
2026-04-08 03:05:58.533656 | TASK [add-build-sshkey : Check to see if ssh key was already created for this build]
2026-04-08 03:05:58.924185 | instance -> localhost | ok
2026-04-08 03:05:58.933621 | 
2026-04-08 03:05:58.933717 | TASK [add-build-sshkey : Create a new key in workspace based on build UUID]
2026-04-08 03:05:58.968544 | instance | ok
2026-04-08 03:05:58.981658 | instance | included: /var/lib/zuul/builds/0f45b49b9fed45abac46197190a5b9cd/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml
2026-04-08 03:05:58.987399 | 
2026-04-08 03:05:58.987460 | TASK [add-build-sshkey : Create Temp SSH key]
2026-04-08 03:05:59.562036 | instance -> localhost | Generating public/private rsa key pair.
2026-04-08 03:05:59.562214 | instance -> localhost | Your identification has been saved in /var/lib/zuul/builds/0f45b49b9fed45abac46197190a5b9cd/work/0f45b49b9fed45abac46197190a5b9cd_id_rsa
2026-04-08 03:05:59.562256 | instance -> localhost | Your public key has been saved in /var/lib/zuul/builds/0f45b49b9fed45abac46197190a5b9cd/work/0f45b49b9fed45abac46197190a5b9cd_id_rsa.pub
2026-04-08 03:05:59.562288 | instance -> localhost | The key fingerprint is:
2026-04-08 03:05:59.562319 | instance -> localhost | SHA256:ydnBhLUaai/3/df2Y7BwHCw/IRKRQMD3uamwur5PqoU zuul-build-sshkey
2026-04-08 03:05:59.562360 | instance -> localhost | The key's randomart image is:
2026-04-08 03:05:59.562390 | instance -> localhost | +---[RSA 3072]----+
2026-04-08 03:05:59.562422 | instance -> localhost | |    ..oo.+=      |
2026-04-08 03:05:59.562452 | instance -> localhost | |     . ..= .     |
2026-04-08 03:05:59.562480 | instance -> localhost | |      . o * .    |
2026-04-08 03:05:59.562507 | instance -> localhost | |       o X + +   |
2026-04-08 03:05:59.562535 | instance -> localhost | |      o S = = o  |
2026-04-08 03:05:59.562561 | instance -> localhost | |  .  o . o . *   |
2026-04-08 03:05:59.562591 | instance -> localhost | | E . .+ +   o + .|
2026-04-08 03:05:59.562618 | instance -> localhost | |  . o. + . . . o+|
2026-04-08 03:05:59.562649 | instance -> localhost | | .o*=.    . ..oo+|
2026-04-08 03:05:59.562677 | instance -> localhost | +----[SHA256]-----+
2026-04-08 03:05:59.562738 | instance -> localhost | ok: Runtime: 0:00:00.177703
2026-04-08 03:05:59.569906 | 
2026-04-08 03:05:59.569990 | TASK [add-build-sshkey : Remote setup ssh keys (linux)]
2026-04-08 03:05:59.600752 | instance | ok
2026-04-08 03:05:59.610689 | instance | included: /var/lib/zuul/builds/0f45b49b9fed45abac46197190a5b9cd/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml
2026-04-08 03:05:59.621633 | 
2026-04-08 03:05:59.621753 | TASK [add-build-sshkey : Remove previously added zuul-build-sshkey]
2026-04-08 03:05:59.666369 | instance | skipping: Conditional result was False
2026-04-08 03:05:59.674634 | 
2026-04-08 03:05:59.674724 | TASK [add-build-sshkey : Enable access via build key on all nodes]
2026-04-08 03:06:00.092171 | instance | changed
2026-04-08 03:06:00.152273 | 
2026-04-08 03:06:00.152385 | TASK [add-build-sshkey : Make sure user has a .ssh]
2026-04-08 03:06:00.337466 | instance | ok
2026-04-08 03:06:00.345331 | 
2026-04-08 03:06:00.345419 | TASK [add-build-sshkey : Install build private key as SSH key on all nodes]
2026-04-08 03:06:00.786257 | instance | changed
2026-04-08 03:06:00.793029 | 
2026-04-08 03:06:00.793118 | TASK [add-build-sshkey : Install build public key as SSH key on all nodes]
2026-04-08 03:06:01.226937 | instance | changed
2026-04-08 03:06:01.233744 | 
2026-04-08 03:06:01.233831 | TASK [add-build-sshkey : Remote setup ssh keys (windows)]
2026-04-08 03:06:01.257545 | instance | skipping: Conditional result was False
2026-04-08 03:06:01.267992 | 
2026-04-08 03:06:01.268077 | TASK [remove-zuul-sshkey : Remove master key from local agent]
2026-04-08 03:06:01.637576 | instance -> localhost | changed
2026-04-08 03:06:01.697740 | 
2026-04-08 03:06:01.697894 | TASK [add-build-sshkey : Add back temp key]
2026-04-08 03:06:01.975313 | instance -> localhost | Identity added: /var/lib/zuul/builds/0f45b49b9fed45abac46197190a5b9cd/work/0f45b49b9fed45abac46197190a5b9cd_id_rsa (zuul-build-sshkey)
2026-04-08 03:06:01.975640 | instance -> localhost | ok: Runtime: 0:00:00.013617
2026-04-08 03:06:01.980947 | 
2026-04-08 03:06:01.981077 | TASK [add-build-sshkey : Verify we can still SSH to all nodes]
2026-04-08 03:06:02.254998 | instance | ok
2026-04-08 03:06:02.262157 | 
2026-04-08 03:06:02.262316 | TASK [add-build-sshkey : Verify we can still SSH to all nodes (windows)]
2026-04-08 03:06:02.286522 | instance | skipping: Conditional result was False
2026-04-08 03:06:02.301139 | 
2026-04-08 03:06:02.301296 | TASK [prepare-workspace : Start zuul_console daemon.]
2026-04-08 03:06:02.576787 | instance | ok
2026-04-08 03:06:02.582807 | 
2026-04-08 03:06:02.582904 | TASK [prepare-workspace : Synchronize src repos to workspace directory.]
2026-04-08 03:06:04.173990 | instance | Output suppressed because no_log was given
2026-04-08 03:06:04.185499 | 
2026-04-08 03:06:04.185566 | LOOP [ensure-output-dirs : Empty Zuul Output directories by removing them]
2026-04-08 03:06:04.368288 | instance | ok: "logs"
2026-04-08 03:06:04.368518 | instance | ok: All items complete
2026-04-08 03:06:04.368555 | 
2026-04-08 03:06:04.524741 | instance | ok: "artifacts"
2026-04-08 03:06:04.681680 | instance | ok: "docs"
2026-04-08 03:06:04.697579 | 
2026-04-08 03:06:04.697726 | LOOP [ensure-output-dirs : Ensure Zuul Output directories exist]
2026-04-08 03:06:04.886757 | instance | changed: "logs"
2026-04-08 03:06:05.028208 | instance | changed: "artifacts"
2026-04-08 03:06:05.183614 | instance | changed: "docs"
2026-04-08 03:06:05.205405 | 
2026-04-08 03:06:05.205528 | PLAY RECAP
2026-04-08 03:06:05.205590 | instance | ok: 15 changed: 8 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0
2026-04-08 03:06:05.205629 | localhost | ok: 6 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0
2026-04-08 03:06:05.205715 | 
2026-04-08 03:06:05.317969 | PRE-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main]
2026-04-08 03:06:05.327739 | PRE-RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main]
2026-04-08 03:06:05.907296 | 
2026-04-08 03:06:05.907425 | PLAY [all]
2026-04-08 03:06:05.920201 | 
2026-04-08 03:06:05.920328 | TASK [setup-uv : Extract archive]
2026-04-08 03:06:08.091901 | instance | changed
2026-04-08 03:06:08.099570 | 
2026-04-08 03:06:08.099674 | TASK [setup-uv : Print version]
2026-04-08 03:06:08.781119 | instance | uv 0.8.13
2026-04-08 03:06:08.652921 | instance | ok: Runtime: 0:00:00.012327
2026-04-08 03:06:08.661029 | 
2026-04-08 03:06:08.661099 | PLAY RECAP
2026-04-08 03:06:08.661163 | instance | ok: 2 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0
2026-04-08 03:06:08.661196 | 
2026-04-08 03:06:08.764492 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main]
2026-04-08 03:06:08.774557 | PRE-RUN START: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/pre.yml@main]
2026-04-08 03:06:09.344805 | 
2026-04-08 03:06:09.344912 | PLAY [all]
2026-04-08 03:06:09.355868 | 
2026-04-08 03:06:09.355939 | TASK [Install "jq" for log collection]
2026-04-08 03:06:19.065845 | instance | changed
2026-04-08 03:06:19.068611 | 
2026-04-08 03:06:19.068665 | PLAY RECAP
2026-04-08 03:06:19.068715 | instance | ok: 1 changed: 1 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0
2026-04-08 03:06:19.068762 | 
2026-04-08 03:06:19.154700 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/pre.yml@main]
2026-04-08 03:06:19.165840 | RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/run.yaml@main]
2026-04-08 03:06:19.710425 | 
2026-04-08 03:06:19.710545 | PLAY [all]
2026-04-08 03:06:19.721195 | 
2026-04-08 03:06:19.721265 | TASK [Copy inventory file for Zuul]
2026-04-08 03:06:20.503289 | instance | changed
2026-04-08 03:06:20.508040 | 
2026-04-08 03:06:20.508109 | TASK [Switch "ansible_host" to private IP]
2026-04-08 03:06:20.803496 | instance | changed: 1 replacements made
2026-04-08 03:06:20.808500 | 
2026-04-08 03:06:20.808560 | TASK [Run Molecule scenario]
2026-04-08 03:06:21.285494 | instance | Using CPython 3.10.12 interpreter at: /usr/bin/python3
2026-04-08 03:06:21.285682 | instance | Creating virtual environment at: .venv
2026-04-08 03:06:21.311246 | instance |    Building atmosphere @ file:///home/zuul/src/github.com/vexxhost/atmosphere
2026-04-08 03:06:21.330750 | instance | Downloading openstacksdk (1.7MiB)
2026-04-08 03:06:21.330914 | instance | Downloading netaddr (2.2MiB)
2026-04-08 03:06:21.331019 | instance | Downloading setuptools (1.1MiB)
2026-04-08 03:06:21.331184 | instance | Downloading cryptography (4.2MiB)
2026-04-08 03:06:21.331330 | instance | Downloading pygments (1.2MiB)
2026-04-08 03:06:21.331604 | instance | Downloading ansible-core (2.1MiB)
2026-04-08 03:06:21.331900 | instance | Downloading pydantic-core (2.0MiB)
2026-04-08 03:06:21.332599 | instance | Downloading kubernetes (1.9MiB)
2026-04-08 03:06:21.341874 | instance | Downloading rjsonnet (1.2MiB)
2026-04-08 03:06:21.676139 | instance |    Building pyperclip==1.9.0
2026-04-08 03:06:21.699100 | instance |  Downloading rjsonnet
2026-04-08 03:06:21.792925 | instance |  Downloading pydantic-core
2026-04-08 03:06:21.833773 | instance |  Downloading netaddr
2026-04-08 03:06:21.851064 | instance |  Downloading pygments
2026-04-08 03:06:21.865295 | instance |  Downloading cryptography
2026-04-08 03:06:21.900699 | instance |  Downloading setuptools
2026-04-08 03:06:21.964011 | instance |  Downloading kubernetes
2026-04-08 03:06:22.000512 | instance |  Downloading ansible-core
2026-04-08 03:06:22.036089 | instance |  Downloading openstacksdk
2026-04-08 03:06:22.359621 | instance |       Built pyperclip==1.9.0
2026-04-08 03:06:22.565140 | instance |       Built atmosphere @ file:///home/zuul/src/github.com/vexxhost/atmosphere
2026-04-08 03:06:22.604568 | instance | Installed 83 packages in 37ms
2026-04-08 03:06:23.221468 | instance | WARNING  Molecule scenarios should migrate to 'extensions/molecule'
2026-04-08 03:06:23.819732 | instance | INFO     [aio > discovery] scenario test matrix: dependency, cleanup, destroy, syntax, create, prepare, converge, idempotence, side_effect, verify, cleanup, destroy
2026-04-08 03:06:23.819868 | instance | INFO     [aio > prerun] Performing prerun with role_name_check=0...
2026-04-08 03:07:06.000818 | instance | INFO     [aio > dependency] Executing
2026-04-08 03:07:06.000940 | instance | WARNING  [aio > dependency] Missing roles requirements file: requirements.yml
2026-04-08 03:07:06.001171 | instance | WARNING  [aio > dependency] Missing collections requirements file: collections.yml
2026-04-08 03:07:06.001274 | instance | WARNING  [aio > dependency] Executed: 2 missing (Remove from test_sequence to suppress)
2026-04-08 03:07:06.011276 | instance | INFO     [aio > cleanup] Executing
2026-04-08 03:07:06.011573 | instance | WARNING  [aio > cleanup] Executed: Missing playbook (Remove from test_sequence to suppress)
2026-04-08 03:07:06.022266 | instance | INFO     [aio > destroy] Executing
2026-04-08 03:07:06.022309 | instance | WARNING  [aio > destroy] Skipping, '--destroy=never' requested.
2026-04-08 03:07:06.022394 | instance | INFO     [aio > destroy] Executed: Successful
2026-04-08 03:07:06.033654 | instance | INFO     [aio > syntax] Executing
2026-04-08 03:07:08.969965 | instance |
2026-04-08 03:07:08.970262 | instance | playbook: /home/zuul/src/github.com/vexxhost/atmosphere/molecule/aio/converge.yml
2026-04-08 03:07:09.089262 | instance | INFO     [aio > syntax] Executed: Successful
2026-04-08 03:07:09.106491 | instance | INFO     [aio > create] Executing
2026-04-08 03:07:09.108426 | instance | WARNING  [aio > create] Executed: Missing playbook (Remove from test_sequence to suppress)
2026-04-08 03:07:09.118634 | instance | INFO     [aio > prepare] Executing
2026-04-08 03:07:09.962167 | instance |
2026-04-08 03:07:09.962284 | instance | PLAY [Prepare] *****************************************************************
2026-04-08 03:07:09.962486 | instance |
2026-04-08 03:07:09.962719 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 03:07:09.962951 | instance | Wednesday 08 April 2026  03:07:09 +0000 (0:00:00.030)       0:00:00.030 *******
2026-04-08 03:07:11.292760 | instance | [WARNING]: Platform linux on host instance is using the discovered Python
2026-04-08 03:07:11.293045 | instance | interpreter at /usr/bin/python3.10, but future installation of another Python
2026-04-08 03:07:11.293339 | instance | interpreter could change the meaning of that path. See
2026-04-08 03:07:11.293645 | instance | https://docs.ansible.com/ansible-
2026-04-08 03:07:11.293945 | instance | core/2.17/reference_appendices/interpreter_discovery.html for more information.
2026-04-08 03:07:11.303186 | instance | ok: [instance]
2026-04-08 03:07:11.303541 | instance |
2026-04-08 03:07:11.303874 | instance | TASK [Configure short hostname] ************************************************
2026-04-08 03:07:11.304151 | instance | Wednesday 08 April 2026  03:07:11 +0000 (0:00:01.341)       0:00:01.371 *******
2026-04-08 03:07:12.108932 | instance | changed: [instance]
2026-04-08 03:07:12.109034 | instance |
2026-04-08 03:07:12.109144 | instance | TASK [Ensure hostname inside hosts file] ***************************************
2026-04-08 03:07:12.109267 | instance | Wednesday 08 April 2026  03:07:12 +0000 (0:00:00.806)       0:00:02.177 *******
2026-04-08 03:07:12.477370 | instance | changed: [instance]
2026-04-08 03:07:12.477669 | instance |
2026-04-08 03:07:12.478002 | instance | TASK [Install "dirmngr" for GPG keyserver operations] **************************
2026-04-08 03:07:12.478319 | instance | Wednesday 08 April 2026  03:07:12 +0000 (0:00:00.368)       0:00:02.545 *******
2026-04-08 03:07:13.825265 | instance | ok: [instance]
2026-04-08 03:07:13.825530 | instance |
2026-04-08 03:07:13.825812 | instance | TASK [Purge "snapd" package] ***************************************************
2026-04-08 03:07:13.826088 | instance | Wednesday 08 April 2026  03:07:13 +0000 (0:00:01.347)       0:00:03.893 *******
2026-04-08 03:07:14.633600 | instance | ok: [instance]
2026-04-08 03:07:14.633706 | instance |
2026-04-08 03:07:14.633898 | instance | PLAY [Generate workspace for Atmosphere] ***************************************
2026-04-08 03:07:14.634062 | instance |
2026-04-08 03:07:14.634239 | instance | TASK [Create folders for workspace] ********************************************
2026-04-08 03:07:14.634504 | instance | Wednesday 08 April 2026  03:07:14 +0000 (0:00:00.809)       0:00:04.702 *******
2026-04-08 03:07:15.891963 | instance | changed: [localhost] => (item=group_vars)
2026-04-08 03:07:15.892094 | instance | changed: [localhost] => (item=group_vars/all)
2026-04-08 03:07:15.892344 | instance | changed: [localhost] => (item=group_vars/controllers)
2026-04-08 03:07:15.892574 | instance | changed: [localhost] => (item=group_vars/cephs)
2026-04-08 03:07:15.892795 | instance | changed: [localhost] => (item=group_vars/computes)
2026-04-08 03:07:15.893021 | instance | changed: [localhost] => (item=host_vars)
2026-04-08 03:07:15.893263 | instance |
2026-04-08 03:07:15.893461 | instance | PLAY [Generate Ceph control plane configuration for workspace] *****************
2026-04-08 03:07:15.893671 | instance |
2026-04-08 03:07:15.893949 | instance | TASK [Ensure the Ceph control plane configuration file exists] *****************
2026-04-08 03:07:15.894183 | instance | Wednesday 08 April 2026  03:07:15 +0000 (0:00:01.257)       0:00:05.959 *******
2026-04-08 03:07:16.097565 | instance | changed: [localhost]
2026-04-08 03:07:16.097794 | instance |
2026-04-08 03:07:16.098032 | instance | TASK [Load the current Ceph control plane configuration into a variable] *******
2026-04-08 03:07:16.098273 | instance | Wednesday 08 April 2026  03:07:16 +0000 (0:00:00.205)       0:00:06.165 *******
2026-04-08 03:07:16.133939 | instance | ok: [localhost]
2026-04-08 03:07:16.134272 | instance |
2026-04-08 03:07:16.134636 | instance | TASK [Generate Ceph control plane values for missing variables] ****************
2026-04-08 03:07:16.134984 | instance | Wednesday 08 April 2026  03:07:16 +0000 (0:00:00.036)       0:00:06.201 *******
2026-04-08 03:07:16.184344 | instance | ok: [localhost] => (item={'key': 'ceph_fsid', 'value': 'd3a1a6c8-e489-5155-9935-be21906f6678'})
2026-04-08 03:07:16.184531 | instance | ok: [localhost] => (item={'key': 'ceph_mon_public_network', 'value': '10.96.240.0/24'})
2026-04-08 03:07:16.184665 | instance |
2026-04-08 03:07:16.184834 | instance | TASK [Write new Ceph control plane configuration file to disk] *****************
2026-04-08 03:07:16.184994 | instance | Wednesday 08 April 2026  03:07:16 +0000 (0:00:00.050)       0:00:06.252 *******
2026-04-08 03:07:16.777266 | instance | changed: [localhost]
2026-04-08 03:07:16.777545 | instance |
2026-04-08 03:07:16.777822 | instance | PLAY [Generate Ceph OSD configuration for workspace] ***************************
2026-04-08 03:07:16.778084 | instance |
2026-04-08 03:07:16.778417 | instance | TASK [Ensure the Ceph OSDs configuration file exists] **************************
2026-04-08 03:07:16.778688 | instance | Wednesday 08 April 2026  03:07:16 +0000 (0:00:00.592)       0:00:06.845 *******
2026-04-08 03:07:16.995067 | instance | changed: [localhost]
2026-04-08 03:07:16.995262 | instance |
2026-04-08 03:07:16.995537 | instance | TASK [Load the current Ceph OSDs configuration into a variable] ****************
2026-04-08 03:07:16.995985 | instance | Wednesday 08 April 2026  03:07:16 +0000 (0:00:00.217)       0:00:07.063 *******
2026-04-08 03:07:17.029429 | instance | ok: [localhost]
2026-04-08 03:07:17.029671 | instance |
2026-04-08 03:07:17.029961 | instance | TASK [Generate Ceph OSDs values for missing variables] *************************
2026-04-08 03:07:17.030249 | instance | Wednesday 08 April 2026  03:07:17 +0000 (0:00:00.034)       0:00:07.097 *******
2026-04-08 03:07:17.068027 | instance | ok: [localhost] => (item={'key': 'ceph_osd_devices', 'value': ['/dev/vdb', '/dev/vdc', '/dev/vdd']})
2026-04-08 03:07:17.068300 | instance |
2026-04-08 03:07:17.068634 | instance | TASK [Write new Ceph OSDs configuration file to disk] **************************
2026-04-08 03:07:17.068915 | instance | Wednesday 08 April 2026  03:07:17 +0000 (0:00:00.038)       0:00:07.136 *******
2026-04-08 03:07:17.444248 | instance | changed: [localhost]
2026-04-08 03:07:17.444578 | instance |
2026-04-08 03:07:17.444994 | instance | PLAY [Generate Kubernetes configuration for workspace] *************************
2026-04-08 03:07:17.445309 | instance |
2026-04-08 03:07:17.445558 | instance | TASK [Ensure the Kubernetes configuration file exists] *************************
2026-04-08 03:07:17.445804 | instance | Wednesday 08 April 2026  03:07:17 +0000 (0:00:00.375)       0:00:07.511 *******
2026-04-08 03:07:17.660458 | instance | changed: [localhost]
2026-04-08 03:07:17.660796 | instance |
2026-04-08 03:07:17.661217 | instance | TASK [Load the current Kubernetes configuration into a variable] ***************
2026-04-08 03:07:17.661569 | instance | Wednesday 08 April 2026  03:07:17 +0000 (0:00:00.216)       0:00:07.728 *******
2026-04-08 03:07:17.698924 | instance | ok: [localhost]
2026-04-08 03:07:17.699194 | instance |
2026-04-08 03:07:17.699486 | instance | TASK [Generate Kubernetes values for missing variables] ************************
2026-04-08 03:07:17.699778 | instance | Wednesday 08 April 2026  03:07:17 +0000 (0:00:00.038)       0:00:07.766 *******
2026-04-08 03:07:17.744387 | instance | ok: [localhost] => (item={'key': 'kubernetes_hostname', 'value': '10.96.240.10'})
2026-04-08 03:07:17.744519 | instance | ok: [localhost] => (item={'key': 'kubernetes_keepalived_vrid', 'value': 42})
2026-04-08 03:07:17.744692 | instance | ok: [localhost] => (item={'key': 'kubernetes_keepalived_vip', 'value': '10.96.240.10'})
2026-04-08 03:07:17.744855 | instance |
2026-04-08 03:07:17.745135 | instance | TASK [Write new Kubernetes configuration file to disk] *************************
2026-04-08 03:07:17.745376 | instance | Wednesday 08 April 2026  03:07:17 +0000 (0:00:00.045)       0:00:07.812 *******
2026-04-08 03:07:18.127141 | instance | changed: [localhost]
2026-04-08 03:07:18.127380 | instance |
2026-04-08 03:07:18.127659 | instance | PLAY [Generate Keepalived configuration for workspace] *************************
2026-04-08 03:07:18.127909 | instance |
2026-04-08 03:07:18.128209 | instance | TASK [Ensure the Keeaplived configuration file exists] *************************
2026-04-08 03:07:18.128527 | instance | Wednesday 08 April 2026  03:07:18 +0000 (0:00:00.383)       0:00:08.195 *******
2026-04-08 03:07:18.345365 | instance | changed: [localhost]
2026-04-08 03:07:18.345619 | instance |
2026-04-08 03:07:18.345921 | instance | TASK [Load the current Keepalived configuration into a variable] ***************
2026-04-08 03:07:18.346221 | instance | Wednesday 08 April 2026  03:07:18 +0000 (0:00:00.217)       0:00:08.413 *******
2026-04-08 03:07:18.379691 | instance | ok: [localhost]
2026-04-08 03:07:18.380028 | instance |
2026-04-08 03:07:18.380356 | instance | TASK [Generate Keepalived values for missing variables] ************************
2026-04-08 03:07:18.380701 | instance | Wednesday 08 April 2026  03:07:18 +0000 (0:00:00.034)       0:00:08.447 *******
2026-04-08 03:07:18.414466 | instance | ok: [localhost] => (item={'key': 'keepalived_interface', 'value': 'br-ex'})
2026-04-08 03:07:18.414713 | instance | ok: [localhost] => (item={'key': 'keepalived_vip', 'value': '10.96.250.10'})
2026-04-08 03:07:18.414958 | instance |
2026-04-08 03:07:18.415228 | instance | TASK [Write new Keepalived configuration file to disk] *************************
2026-04-08 03:07:18.415497 | instance | Wednesday 08 April 2026  03:07:18 +0000 (0:00:00.034)       0:00:08.482 *******
2026-04-08 03:07:18.803480 | instance | changed: [localhost]
2026-04-08 03:07:18.803700 | instance |
2026-04-08 03:07:18.803994 | instance | PLAY [Generate endpoints for workspace] ****************************************
2026-04-08 03:07:18.804272 | instance |
2026-04-08 03:07:18.804664 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 03:07:18.804934 | instance | Wednesday 08 April 2026  03:07:18 +0000 (0:00:00.388)       0:00:08.871 *******
2026-04-08 03:07:19.547005 | instance | ok: [localhost]
2026-04-08 03:07:19.547086 | instance |
2026-04-08 03:07:19.547168 | instance | TASK [Ensure the endpoints file exists] ****************************************
2026-04-08 03:07:19.547301 | instance | Wednesday 08 April 2026  03:07:19 +0000 (0:00:00.743)       0:00:09.615 *******
2026-04-08 03:07:19.770629 | instance | changed: [localhost]
2026-04-08 03:07:19.770791 | instance |
2026-04-08 03:07:19.771078 | instance | TASK [Load the current endpoints into a variable] ******************************
2026-04-08 03:07:19.771348 | instance | Wednesday 08 April 2026  03:07:19 +0000 (0:00:00.222)       0:00:09.838 *******
2026-04-08 03:07:19.811398 | instance | ok: [localhost]
2026-04-08 03:07:19.811638 | instance |
2026-04-08 03:07:19.811913 | instance | TASK [Generate endpoint skeleton for missing variables] ************************
2026-04-08 03:07:19.812226 | instance | Wednesday 08 April 2026  03:07:19 +0000 (0:00:00.041)       0:00:09.879 *******
2026-04-08 03:07:20.594103 | instance | ok: [localhost] => (item=keycloak_host)
2026-04-08 03:07:20.594474 | instance | ok: [localhost] => (item=kube_prometheus_stack_grafana_host)
2026-04-08 03:07:20.594714 | instance | ok: [localhost] => (item=kube_prometheus_stack_alertmanager_host)
2026-04-08 03:07:20.594989 | instance | ok: [localhost] => (item=kube_prometheus_stack_prometheus_host)
2026-04-08 03:07:20.595263 | instance | ok: [localhost] => (item=openstack_helm_endpoints_region_name)
2026-04-08 03:07:20.595534 | instance | ok: [localhost] => (item=openstack_helm_endpoints_keystone_api_host)
2026-04-08 03:07:20.595951 | instance | ok: [localhost] => (item=openstack_helm_endpoints_glance_api_host)
2026-04-08 03:07:20.596232 | instance | ok: [localhost] => (item=openstack_helm_endpoints_cinder_api_host)
2026-04-08 03:07:20.596622 | instance | ok: [localhost] => (item=openstack_helm_endpoints_placement_api_host)
2026-04-08 03:07:20.596915 | instance | ok: [localhost] => (item=openstack_helm_endpoints_barbican_api_host)
2026-04-08 03:07:20.597189 | instance | ok: [localhost] => (item=openstack_helm_endpoints_neutron_api_host)
2026-04-08 03:07:20.597411 | instance | ok: [localhost] => (item=openstack_helm_endpoints_nova_api_host)
2026-04-08 03:07:20.597667 | instance | ok: [localhost] => (item=openstack_helm_endpoints_nova_novnc_host)
2026-04-08 03:07:20.597935 | instance | ok: [localhost] => (item=openstack_helm_endpoints_ironic_api_host)
2026-04-08 03:07:20.598202 | instance | ok: [localhost] => (item=openstack_helm_endpoints_designate_api_host)
2026-04-08 03:07:20.598470 | instance | ok: [localhost] => (item=openstack_helm_endpoints_octavia_api_host)
2026-04-08 03:07:20.598735 | instance | ok: [localhost] => (item=openstack_helm_endpoints_magnum_api_host)
2026-04-08 03:07:20.599006 | instance | ok: [localhost] => (item=openstack_helm_endpoints_magnum_registry_host)
2026-04-08 03:07:20.599269 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_api_host)
2026-04-08 03:07:20.599534 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_cfn_api_host)
2026-04-08 03:07:20.599828 | instance | ok: [localhost] => (item=openstack_helm_endpoints_horizon_api_host)
2026-04-08 03:07:20.600093 | instance | ok: [localhost] => (item=openstack_helm_endpoints_rgw_host)
2026-04-08 03:07:20.600418 | instance | ok: [localhost] => (item=openstack_helm_endpoints_manila_api_host)
2026-04-08 03:07:20.600687 | instance |
2026-04-08 03:07:20.600960 | instance | TASK [Write new endpoints file to disk] ****************************************
2026-04-08 03:07:20.601227 | instance | Wednesday 08 April 2026  03:07:20 +0000 (0:00:00.782)       0:00:10.662 *******
2026-04-08 03:07:20.962371 | instance | changed: [localhost]
2026-04-08 03:07:20.962612 | instance |
2026-04-08 03:07:20.962927 | instance | TASK [Ensure the endpoints file exists] ****************************************
2026-04-08 03:07:20.963245 | instance | Wednesday 08 April 2026  03:07:20 +0000 (0:00:00.368)       0:00:11.030 *******
2026-04-08 03:07:21.181133 | instance | changed: [localhost]
2026-04-08 03:07:21.181386 | instance |
2026-04-08 03:07:21.181701 | instance | PLAY [Generate Neutron configuration for workspace] ****************************
2026-04-08 03:07:21.181978 | instance |
2026-04-08 03:07:21.182278 | instance | TASK [Ensure the Neutron configuration file exists] ****************************
2026-04-08 03:07:21.182579 | instance | Wednesday 08 April 2026  03:07:21 +0000 (0:00:00.218)       0:00:11.249 *******
2026-04-08 03:07:21.414318 | instance | changed: [localhost]
2026-04-08 03:07:21.414465 | instance |
2026-04-08 03:07:21.414806 | instance | TASK [Load the current Neutron configuration into a variable] ******************
2026-04-08 03:07:21.415087 | instance | Wednesday 08 April 2026  03:07:21 +0000 (0:00:00.233)       0:00:11.482 *******
2026-04-08 03:07:21.451661 | instance | ok: [localhost]
2026-04-08 03:07:21.451785 | instance |
2026-04-08 03:07:21.452019 | instance | TASK [Generate Neutron values for missing variables] ***************************
2026-04-08 03:07:21.452145 | instance | Wednesday 08 April 2026  03:07:21 +0000 (0:00:00.037)       0:00:11.520 *******
2026-04-08 03:07:21.488967 | instance | ok: [localhost] => (item={'key': 'neutron_networks', 'value': [{'name': 'public', 'external': True, 'shared': True, 'mtu_size': 1500, 'port_security_enabled': True, 'provider_network_type': 'flat', 'provider_physical_network': 'external', 'subnets': [{'name': 'public-subnet', 'cidr': '10.96.250.0/24', 'gateway_ip': '10.96.250.10', 'allocation_pool_start': '10.96.250.200', 'allocation_pool_end': '10.96.250.220', 'enable_dhcp': True}]}]})
2026-04-08 03:07:21.489026 | instance |
2026-04-08 03:07:21.489158 | instance | TASK [Write new Neutron configuration file to disk] ****************************
2026-04-08 03:07:21.489326 | instance | Wednesday 08 April 2026  03:07:21 +0000 (0:00:00.037)       0:00:11.557 *******
2026-04-08 03:07:21.887802 | instance | changed: [localhost]
2026-04-08 03:07:21.888045 | instance |
2026-04-08 03:07:21.888401 | instance | PLAY [Generate Nova configuration for workspace] *******************************
2026-04-08 03:07:21.888676 | instance |
2026-04-08 03:07:21.888953 | instance | TASK [Ensure the Nova configuration file exists] *******************************
2026-04-08 03:07:21.889233 | instance | Wednesday 08 April 2026  03:07:21 +0000 (0:00:00.398)       0:00:11.956 *******
2026-04-08 03:07:22.114877 | instance | changed: [localhost]
2026-04-08 03:07:22.115139 | instance |
2026-04-08 03:07:22.115416 | instance | TASK [Load the current Nova configuration into a variable] *********************
2026-04-08 03:07:22.115722 | instance | Wednesday 08 April 2026  03:07:22 +0000 (0:00:00.226)       0:00:12.183 *******
2026-04-08 03:07:22.149750 | instance | ok: [localhost]
2026-04-08 03:07:22.149847 | instance |
2026-04-08 03:07:22.149970 | instance | TASK [Generate Nova values for missing variables] ******************************
2026-04-08 03:07:22.150091 | instance | Wednesday 08 April 2026  03:07:22 +0000 (0:00:00.035)       0:00:12.218 *******
2026-04-08 03:07:22.188301 | instance | ok: [localhost] => (item={'key': 'nova_flavors', 'value': [{'name': 'm1.tiny', 'ram': 512, 'disk': 1, 'vcpus': 1}, {'name': 'm1.small', 'ram': 2048, 'disk': 20, 'vcpus': 1}, {'name': 'm1.medium', 'ram': 4096, 'disk': 40, 'vcpus': 2}, {'name': 'm1.large', 'ram': 8192, 'disk': 80, 'vcpus': 4}, {'name': 'm1.xlarge', 'ram': 16384, 'disk': 160, 'vcpus': 8}]})
2026-04-08 03:07:22.188435 | instance |
2026-04-08 03:07:22.188604 | instance | TASK [Write new Nova configuration file to disk] *******************************
2026-04-08 03:07:22.188774 | instance | Wednesday 08 April 2026  03:07:22 +0000 (0:00:00.038)       0:00:12.256 *******
2026-04-08 03:07:22.548624 | instance | changed: [localhost]
2026-04-08 03:07:22.548909 | instance |
2026-04-08 03:07:22.549311 | instance | PLAY [Generate secrets for workspace] ******************************************
2026-04-08 03:07:22.549625 | instance |
2026-04-08 03:07:22.549942 | instance | TASK [Ensure the secrets file exists] ******************************************
2026-04-08 03:07:22.550492 | instance | Wednesday 08 April 2026  03:07:22 +0000 (0:00:00.360)       0:00:12.616 *******
2026-04-08 03:07:22.779915 | instance | changed: [localhost]
2026-04-08 03:07:22.780168 | instance |
2026-04-08 03:07:22.780524 | instance | TASK [Load the current secrets into a variable] ********************************
2026-04-08 03:07:22.780893 | instance | Wednesday 08 April 2026  03:07:22 +0000 (0:00:00.231)       0:00:12.848 *******
2026-04-08 03:07:22.815259 | instance | ok: [localhost]
2026-04-08 03:07:22.815339 | instance |
2026-04-08 03:07:22.815550 | instance | TASK [Generate secrets for missing variables] **********************************
2026-04-08 03:07:22.815724 | instance | Wednesday 08 April 2026  03:07:22 +0000 (0:00:00.035)       0:00:12.883 *******
2026-04-08 03:07:23.213326 | instance | ok: [localhost] => (item=heat_auth_encryption_key)
2026-04-08 03:07:23.213859 | instance | ok: [localhost] => (item=keepalived_password)
2026-04-08 03:07:23.214199 | instance | ok: [localhost] => (item=keycloak_admin_password)
2026-04-08 03:07:23.214564 | instance | ok: [localhost] => (item=keycloak_database_password)
2026-04-08 03:07:23.214893 | instance | ok: [localhost] => (item=keystone_keycloak_client_secret)
2026-04-08 03:07:23.215219 | instance | ok: [localhost] => (item=keystone_oidc_crypto_passphrase)
2026-04-08 03:07:23.215544 | instance | ok: [localhost] => (item=kube_prometheus_stack_grafana_admin_password)
2026-04-08 03:07:23.215865 | instance | ok: [localhost] => (item=octavia_heartbeat_key)
2026-04-08 03:07:23.216188 | instance | ok: [localhost] => (item=openstack_helm_endpoints_rabbitmq_admin_password)
2026-04-08 03:07:23.216568 | instance | ok: [localhost] => (item=openstack_helm_endpoints_memcached_secret_key)
2026-04-08 03:07:23.216899 | instance | ok: [localhost] => (item=openstack_helm_endpoints_keystone_admin_password)
2026-04-08 03:07:23.217223 | instance | ok: [localhost] => (item=openstack_helm_endpoints_keystone_mariadb_password)
2026-04-08 03:07:23.217546 | instance | ok: [localhost] => (item=openstack_helm_endpoints_keystone_rabbitmq_password)
2026-04-08 03:07:23.217869 | instance | ok: [localhost] => (item=openstack_helm_endpoints_glance_keystone_password)
2026-04-08 03:07:23.218190 | instance | ok: [localhost] => (item=openstack_helm_endpoints_glance_mariadb_password)
2026-04-08 03:07:23.218537 | instance | ok: [localhost] => (item=openstack_helm_endpoints_glance_rabbitmq_password)
2026-04-08 03:07:23.218860 | instance | ok: [localhost] => (item=openstack_helm_endpoints_cinder_keystone_password)
2026-04-08 03:07:23.219182 | instance | ok: [localhost] => (item=openstack_helm_endpoints_cinder_mariadb_password)
2026-04-08 03:07:23.219526 | instance | ok: [localhost] => (item=openstack_helm_endpoints_cinder_rabbitmq_password)
2026-04-08 03:07:23.219875 | instance | ok: [localhost] => (item=openstack_helm_endpoints_placement_keystone_password)
2026-04-08 03:07:23.220222 | instance | ok: [localhost] => (item=openstack_helm_endpoints_placement_mariadb_password)
2026-04-08 03:07:23.220604 | instance | ok: [localhost] => (item=openstack_helm_endpoints_barbican_keystone_password)
2026-04-08 03:07:23.220956 | instance | ok: [localhost] => (item=openstack_helm_endpoints_barbican_mariadb_password)
2026-04-08 03:07:23.221303 | instance | ok: [localhost] => (item=openstack_helm_endpoints_neutron_keystone_password)
2026-04-08 03:07:23.221655 | instance | ok: [localhost] => (item=openstack_helm_endpoints_neutron_mariadb_password)
2026-04-08 03:07:23.222003 | instance | ok: [localhost] => (item=openstack_helm_endpoints_neutron_rabbitmq_password)
2026-04-08 03:07:23.222358 | instance | ok: [localhost] => (item=openstack_helm_endpoints_neutron_metadata_secret)
2026-04-08 03:07:23.222770 | instance | ok: [localhost] => (item=openstack_helm_endpoints_nova_keystone_password)
2026-04-08 03:07:23.223121 | instance | ok: [localhost] => (item=openstack_helm_endpoints_nova_mariadb_password)
2026-04-08 03:07:23.223385 | instance | ok: [localhost] => (item=openstack_helm_endpoints_nova_rabbitmq_password)
2026-04-08 03:07:23.223533 | instance | ok: [localhost] => (item=openstack_helm_endpoints_ironic_keystone_password)
2026-04-08 03:07:23.223680 | instance | ok: [localhost] => (item=openstack_helm_endpoints_ironic_mariadb_password)
2026-04-08 03:07:23.223827 | instance | ok: [localhost] => (item=openstack_helm_endpoints_ironic_rabbitmq_password)
2026-04-08 03:07:23.223974 | instance | ok: [localhost] => (item=openstack_helm_endpoints_designate_keystone_password)
2026-04-08 03:07:23.224121 | instance | ok: [localhost] => (item=openstack_helm_endpoints_designate_mariadb_password)
2026-04-08 03:07:23.224268 | instance | ok: [localhost] => (item=openstack_helm_endpoints_designate_rabbitmq_password)
2026-04-08 03:07:23.224455 | instance | ok: [localhost] => (item=openstack_helm_endpoints_octavia_keystone_password)
2026-04-08 03:07:23.224607 | instance | ok: [localhost] => (item=openstack_helm_endpoints_octavia_mariadb_password)
2026-04-08 03:07:23.224754 | instance | ok: [localhost] => (item=openstack_helm_endpoints_octavia_rabbitmq_password)
2026-04-08 03:07:23.224911 | instance | ok: [localhost] => (item=openstack_helm_endpoints_magnum_keystone_password)
2026-04-08 03:07:23.225059 | instance | ok: [localhost] => (item=openstack_helm_endpoints_magnum_mariadb_password)
2026-04-08 03:07:23.225208 | instance | ok: [localhost] => (item=openstack_helm_endpoints_magnum_rabbitmq_password)
2026-04-08 03:07:23.225355 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_keystone_password)
2026-04-08 03:07:23.225517 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_trustee_keystone_password)
2026-04-08 03:07:23.225671 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_stack_user_keystone_password)
2026-04-08 03:07:23.225819 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_mariadb_password)
2026-04-08 03:07:23.225968 | instance | ok: [localhost] => (item=openstack_helm_endpoints_heat_rabbitmq_password)
2026-04-08 03:07:23.226116 | instance | ok: [localhost] => (item=openstack_helm_endpoints_horizon_mariadb_password)
2026-04-08 03:07:23.226263 | instance | ok: [localhost] => (item=openstack_helm_endpoints_tempest_keystone_password)
2026-04-08 03:07:23.226435 | instance | ok: [localhost] => (item=openstack_helm_endpoints_openstack_exporter_keystone_password)
2026-04-08 03:07:23.226586 | instance | ok: [localhost] => (item=openstack_helm_endpoints_rgw_keystone_password)
2026-04-08 03:07:23.226735 | instance | ok: [localhost] => (item=openstack_helm_endpoints_manila_keystone_password)
2026-04-08 03:07:23.226886 | instance | ok: [localhost] => (item=openstack_helm_endpoints_manila_mariadb_password)
2026-04-08 03:07:23.227037 | instance | ok: [localhost] => (item=openstack_helm_endpoints_staffeln_mariadb_password)
2026-04-08 03:07:23.227180 | instance |
2026-04-08 03:07:23.227330 | instance | TASK [Generate base64 encoded secrets] *****************************************
2026-04-08 03:07:23.227479 | instance | Wednesday 08 April 2026  03:07:23 +0000 (0:00:00.398)       0:00:13.281 *******
2026-04-08 03:07:23.267161 | instance | ok: [localhost] => (item=barbican_kek)
2026-04-08 03:07:23.267332 | instance |
2026-04-08 03:07:23.267492 | instance | TASK [Generate temporary files for generating keys for missing variables] ******
2026-04-08 03:07:23.267652 | instance | Wednesday 08 April 2026  03:07:23 +0000 (0:00:00.052)       0:00:13.334 *******
2026-04-08 03:07:23.723630 | instance | changed: [localhost] => (item=manila_ssh_key)
2026-04-08 03:07:23.723669 | instance | changed: [localhost] => (item=nova_ssh_key)
2026-04-08 03:07:23.723674 | instance |
2026-04-08 03:07:23.723679 | instance | TASK [Generate SSH keys for missing variables] *********************************
2026-04-08 03:07:23.723684 | instance | Wednesday 08 April 2026  03:07:23 +0000 (0:00:00.457)       0:00:13.791 *******
2026-04-08 03:07:26.083461 | instance | changed: [localhost] => (item=manila_ssh_key)
2026-04-08 03:07:26.083498 | instance | changed: [localhost] => (item=nova_ssh_key)
2026-04-08 03:07:26.083503 | instance |
2026-04-08 03:07:26.083508 | instance | TASK [Set values for SSH keys] *************************************************
2026-04-08 03:07:26.083512 | instance | Wednesday 08 April 2026  03:07:26 +0000 (0:00:02.359)       0:00:16.151 *******
2026-04-08 03:07:26.146519 | instance | ok: [localhost] => (item=manila_ssh_key)
2026-04-08 03:07:26.146556 | instance | ok: [localhost] => (item=nova_ssh_key)
2026-04-08 03:07:26.146562 | instance |
2026-04-08 03:07:26.146567 | instance | TASK [Delete the temporary files generated for SSH keys] ***********************
2026-04-08 03:07:26.146571 | instance | Wednesday 08 April 2026  03:07:26 +0000 (0:00:00.062)       0:00:16.214 *******
2026-04-08 03:07:26.525666 | instance | changed: [localhost] => (item=manila_ssh_key)
2026-04-08 03:07:26.525720 | instance | changed: [localhost] => (item=nova_ssh_key)
2026-04-08 03:07:26.525733 | instance |
2026-04-08 03:07:26.525742 | instance | TASK [Write new secrets file to disk] ******************************************
2026-04-08 03:07:26.525751 | instance | Wednesday 08 April 2026  03:07:26 +0000 (0:00:00.378)       0:00:16.592 *******
2026-04-08 03:07:26.907758 | instance | changed: [localhost]
2026-04-08 03:07:26.907793 | instance |
2026-04-08 03:07:26.907800 | instance | TASK [Encrypt secrets file with Vault password] ********************************
2026-04-08 03:07:26.907807 | instance | Wednesday 08 April 2026  03:07:26 +0000 (0:00:00.382)       0:00:16.975 *******
2026-04-08 03:07:26.939484 | instance | skipping: [localhost]
2026-04-08 03:07:26.939531 | instance |
2026-04-08 03:07:26.939538 | instance | PLAY [Overwrite OSD devices with LVM-backed paths] *****************************
2026-04-08 03:07:26.939545 | instance |
2026-04-08 03:07:26.939551 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 03:07:26.939556 | instance | Wednesday 08 April 2026  03:07:26 +0000 (0:00:00.031)       0:00:17.007 *******
2026-04-08 03:07:27.802297 | instance | ok: [instance]
2026-04-08 03:07:27.802334 | instance |
2026-04-08 03:07:27.802339 | instance | TASK [Overwrite existing osds.yml file] ****************************************
2026-04-08 03:07:27.802344 | instance | Wednesday 08 April 2026  03:07:27 +0000 (0:00:00.863)       0:00:17.870 *******
2026-04-08 03:07:28.381379 | instance | changed: [instance]
2026-04-08 03:07:28.381441 | instance |
2026-04-08 03:07:28.381447 | instance | PLAY [Setup networking] ********************************************************
2026-04-08 03:07:28.381452 | instance |
2026-04-08 03:07:28.381457 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 03:07:28.381462 | instance | Wednesday 08 April 2026  03:07:28 +0000 (0:00:00.578)       0:00:18.448 *******
2026-04-08 03:07:29.213337 | instance | ok: [instance]
2026-04-08 03:07:29.213375 | instance |
2026-04-08 03:07:29.213383 | instance | TASK [Create bridge for management network] ************************************
2026-04-08 03:07:29.213390 | instance | Wednesday 08 April 2026  03:07:29 +0000 (0:00:00.832)       0:00:19.280 *******
2026-04-08 03:07:29.689185 | instance | ok: [instance]
2026-04-08 03:07:29.689234 | instance |
2026-04-08 03:07:29.689240 | instance | TASK [Create fake interface for management bridge] *****************************
2026-04-08 03:07:29.689246 | instance | Wednesday 08 April 2026  03:07:29 +0000 (0:00:00.475)       0:00:19.756 *******
2026-04-08 03:07:30.015892 | instance | ok: [instance]
2026-04-08 03:07:30.015937 | instance |
2026-04-08 03:07:30.015945 | instance | TASK [Assign dummy interface to management bridge] *****************************
2026-04-08 03:07:30.015952 | instance | Wednesday 08 April 2026  03:07:30 +0000 (0:00:00.327)       0:00:20.083 *******
2026-04-08 03:07:30.325611 | instance | ok: [instance]
2026-04-08 03:07:30.325654 | instance |
2026-04-08 03:07:30.325660 | instance | TASK [Assign IP address for management bridge] *********************************
2026-04-08 03:07:30.325665 | instance | Wednesday 08 April 2026  03:07:30 +0000 (0:00:00.310)       0:00:20.393 *******
2026-04-08 03:07:30.644029 | instance | ok: [instance]
2026-04-08 03:07:30.644077 | instance |
2026-04-08 03:07:30.644087 | instance | TASK [Bring up interfaces] *****************************************************
2026-04-08 03:07:30.644094 | instance | Wednesday 08 April 2026  03:07:30 +0000 (0:00:00.317)       0:00:20.711 *******
2026-04-08 03:07:31.262869 | instance | ok: [instance] => (item=br-mgmt)
2026-04-08 03:07:31.262907 | instance | ok: [instance] => (item=dummy0)
2026-04-08 03:07:31.262913 | instance |
2026-04-08 03:07:31.262917 | instance | PLAY [Create devices for Ceph] *************************************************
2026-04-08 03:07:31.262922 | instance |
2026-04-08 03:07:31.262926 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 03:07:31.262930 | instance | Wednesday 08 April 2026  03:07:31 +0000 (0:00:00.619)       0:00:21.330 *******
2026-04-08 03:07:32.181551 | instance | ok: [instance]
2026-04-08 03:07:32.181626 | instance |
2026-04-08 03:07:32.182041 | instance | TASK [Install depedencies] *****************************************************
2026-04-08 03:07:32.182083 | instance | Wednesday 08 April 2026  03:07:32 +0000 (0:00:00.918)       0:00:22.249 *******
2026-04-08 03:07:52.473848 | instance | changed: [instance]
2026-04-08 03:07:52.474037 | instance |
2026-04-08 03:07:52.474399 | instance | TASK [Start up service] ********************************************************
2026-04-08 03:07:52.474441 | instance | Wednesday 08 April 2026  03:07:52 +0000 (0:00:20.291)       0:00:42.541 *******
2026-04-08 03:07:53.152949 | instance | ok: [instance]
2026-04-08 03:07:53.153452 | instance |
2026-04-08 03:07:53.153495 | instance | TASK [Generate lvm.conf] *******************************************************
2026-04-08 03:07:53.153503 | instance | Wednesday 08 April 2026  03:07:53 +0000 (0:00:00.679)       0:00:43.221 *******
2026-04-08 03:07:53.477742 | instance | ok: [instance]
2026-04-08 03:07:53.478223 | instance |
2026-04-08 03:07:53.478282 | instance | TASK [Write /etc/lvm/lvm.conf] *************************************************
2026-04-08 03:07:53.478290 | instance | Wednesday 08 April 2026  03:07:53 +0000 (0:00:00.324)       0:00:43.546 *******
2026-04-08 03:07:54.039467 | instance | changed: [instance]
2026-04-08 03:07:54.039577 | instance |
2026-04-08 03:07:54.039948 | instance | TASK [Get list of all loopback devices] ****************************************
2026-04-08 03:07:54.040019 | instance | Wednesday 08 April 2026  03:07:54 +0000 (0:00:00.561)       0:00:44.107 *******
2026-04-08 03:07:54.346523 | instance | ok: [instance]
2026-04-08 03:07:54.346662 | instance |
2026-04-08 03:07:54.346916 | instance | TASK [Fail if there is any existing loopback devices] **************************
2026-04-08 03:07:54.347125 | instance | Wednesday 08 April 2026  03:07:54 +0000 (0:00:00.307)       0:00:44.414 *******
2026-04-08 03:07:54.372656 | instance | skipping: [instance]
2026-04-08 03:07:54.372829 | instance |
2026-04-08 03:07:54.373075 | instance | TASK [Create devices for Ceph] *************************************************
2026-04-08 03:07:54.373301 | instance | Wednesday 08 April 2026  03:07:54 +0000 (0:00:00.025)       0:00:44.440 *******
2026-04-08 03:07:55.204530 | instance | changed: [instance] => (item=osd0)
2026-04-08 03:07:55.204668 | instance | changed: [instance] => (item=osd1)
2026-04-08 03:07:55.204845 | instance | changed: [instance] => (item=osd2)
2026-04-08 03:07:55.205004 | instance |
2026-04-08 03:07:55.205179 | instance | TASK [Set permissions on loopback devices] *************************************
2026-04-08 03:07:55.205355 | instance | Wednesday 08 April 2026  03:07:55 +0000 (0:00:00.831)       0:00:45.272 *******
2026-04-08 03:07:56.057028 | instance | changed: [instance] => (item=osd0)
2026-04-08 03:07:56.057091 | instance | changed: [instance] => (item=osd1)
2026-04-08 03:07:56.057223 | instance | changed: [instance] => (item=osd2)
2026-04-08 03:07:56.057337 | instance |
2026-04-08 03:07:56.057463 | instance | TASK [Start loop devices] ******************************************************
2026-04-08 03:07:56.057589 | instance | Wednesday 08 April 2026  03:07:56 +0000 (0:00:00.852)       0:00:46.125 *******
2026-04-08 03:07:57.098434 | instance | changed: [instance] => (item=osd0)
2026-04-08 03:07:57.098688 | instance | changed: [instance] => (item=osd1)
2026-04-08 03:07:57.099010 | instance | changed: [instance] => (item=osd2)
2026-04-08 03:07:57.099227 | instance |
2026-04-08 03:07:57.099494 | instance | TASK [Create a volume group for each loop device] ******************************
2026-04-08 03:07:57.099869 | instance | Wednesday 08 April 2026  03:07:57 +0000 (0:00:01.040)       0:00:47.166 *******
2026-04-08 03:08:00.550029 | instance | changed: [instance] => (item=osd0)
2026-04-08 03:08:00.550083 | instance | changed: [instance] => (item=osd1)
2026-04-08 03:08:00.550091 | instance | changed: [instance] => (item=osd2)
2026-04-08 03:08:00.550098 | instance |
2026-04-08 03:08:00.550105 | instance | TASK [Create a logical volume for each loop device] ****************************
2026-04-08 03:08:00.550117 | instance | Wednesday 08 April 2026  03:08:00 +0000 (0:00:03.450)       0:00:50.616 *******
2026-04-08 03:08:02.811416 | instance | changed: [instance] => (item=ceph-instance-osd0)
2026-04-08 03:08:02.811495 | instance | changed: [instance] => (item=ceph-instance-osd1)
2026-04-08 03:08:02.811507 | instance | changed: [instance] => (item=ceph-instance-osd2)
2026-04-08 03:08:02.811576 | instance |
2026-04-08 03:08:02.811731 | instance | PLAY [controllers] *************************************************************
2026-04-08 03:08:02.811960 | instance |
2026-04-08 03:08:02.812151 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 03:08:02.812353 | instance | Wednesday 08 April 2026  03:08:02 +0000 (0:00:02.259)       0:00:52.876 *******
2026-04-08 03:08:03.824576 | instance | ok: [instance]
2026-04-08 03:08:03.824698 | instance |
2026-04-08 03:08:03.824711 | instance | TASK [Set masquerade rule] *****************************************************
2026-04-08 03:08:03.824721 | instance | Wednesday 08 April 2026  03:08:03 +0000 (0:00:01.015)       0:00:53.892 *******
2026-04-08 03:08:04.404889 | instance | changed: [instance]
2026-04-08 03:08:04.405103 | instance |
2026-04-08 03:08:04.405407 | instance | PLAY RECAP *********************************************************************
2026-04-08 03:08:04.405720 | instance | instance                   : ok=26   changed=11   unreachable=0    failed=0    skipped=1    rescued=0    ignored=0
2026-04-08 03:08:04.406048 | instance | localhost                  : ok=40   changed=21   unreachable=0    failed=0    skipped=1    rescued=0    ignored=0
2026-04-08 03:08:04.406302 | instance |
2026-04-08 03:08:04.406593 | instance | Wednesday 08 April 2026  03:08:04 +0000 (0:00:00.581)       0:00:54.473 *******
2026-04-08 03:08:04.406871 | instance | ===============================================================================
2026-04-08 03:08:04.407166 | instance | Install depedencies ---------------------------------------------------- 20.29s
2026-04-08 03:08:04.407450 | instance | Create a volume group for each loop device ------------------------------ 3.45s
2026-04-08 03:08:04.407793 | instance | Generate SSH keys for missing variables --------------------------------- 2.36s
2026-04-08 03:08:04.408029 | instance | Create a logical volume for each loop device ---------------------------- 2.26s
2026-04-08 03:08:04.408192 | instance | Install "dirmngr" for GPG keyserver operations -------------------------- 1.35s
2026-04-08 03:08:04.408389 | instance | Gathering Facts --------------------------------------------------------- 1.34s
2026-04-08 03:08:04.408553 | instance | Create folders for workspace -------------------------------------------- 1.26s
2026-04-08 03:08:04.408715 | instance | Start loop devices ------------------------------------------------------ 1.04s
2026-04-08 03:08:04.408888 | instance | Gathering Facts --------------------------------------------------------- 1.02s
2026-04-08 03:08:04.409042 | instance | Gathering Facts --------------------------------------------------------- 0.92s
2026-04-08 03:08:04.409201 | instance | Gathering Facts --------------------------------------------------------- 0.86s
2026-04-08 03:08:04.409365 | instance | Set permissions on loopback devices ------------------------------------- 0.85s
2026-04-08 03:08:04.409528 | instance | Gathering Facts --------------------------------------------------------- 0.83s
2026-04-08 03:08:04.409687 | instance | Create devices for Ceph ------------------------------------------------- 0.83s
2026-04-08 03:08:04.409850 | instance | Purge "snapd" package --------------------------------------------------- 0.81s
2026-04-08 03:08:04.410017 | instance | Configure short hostname ------------------------------------------------ 0.81s
2026-04-08 03:08:04.410176 | instance | Generate endpoint skeleton for missing variables ------------------------ 0.78s
2026-04-08 03:08:04.410345 | instance | Gathering Facts --------------------------------------------------------- 0.74s
2026-04-08 03:08:04.410507 | instance | Start up service -------------------------------------------------------- 0.68s
2026-04-08 03:08:04.410671 | instance | Bring up interfaces ----------------------------------------------------- 0.62s
2026-04-08 03:08:04.529237 | instance | INFO     [aio > prepare] Executed: Successful
2026-04-08 03:08:04.542016 | instance | INFO     [aio > converge] Executing
2026-04-08 03:08:07.291624 | instance |
2026-04-08 03:08:07.291756 | instance | PLAY [all] *********************************************************************
2026-04-08 03:08:07.292060 | instance |
2026-04-08 03:08:07.292362 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 03:08:07.292639 | instance | Wednesday 08 April 2026  03:08:07 +0000 (0:00:00.021)       0:00:00.021 *******
2026-04-08 03:08:08.683232 | instance | [WARNING]: Platform linux on host instance is using the discovered Python
2026-04-08 03:08:08.683454 | instance | interpreter at /usr/bin/python3.10, but future installation of another Python
2026-04-08 03:08:08.683769 | instance | interpreter could change the meaning of that path. See
2026-04-08 03:08:08.684132 | instance | https://docs.ansible.com/ansible-
2026-04-08 03:08:08.684554 | instance | core/2.17/reference_appendices/interpreter_discovery.html for more information.
2026-04-08 03:08:08.693422 | instance | ok: [instance]
2026-04-08 03:08:08.693672 | instance |
2026-04-08 03:08:08.693947 | instance | TASK [Fail if atmosphere_ceph_enabled is set] **********************************
2026-04-08 03:08:08.694246 | instance | Wednesday 08 April 2026  03:08:08 +0000 (0:00:01.402)       0:00:01.423 *******
2026-04-08 03:08:08.736516 | instance | skipping: [instance]
2026-04-08 03:08:08.736894 | instance |
2026-04-08 03:08:08.737293 | instance | TASK [Set a fact with the "atmosphere_images" for other plays] *****************
2026-04-08 03:08:08.737577 | instance | Wednesday 08 April 2026  03:08:08 +0000 (0:00:00.042)       0:00:01.466 *******
2026-04-08 03:08:08.914935 | instance | ok: [instance]
2026-04-08 03:08:08.915242 | instance |
2026-04-08 03:08:08.915581 | instance | PLAY [Deploy Ceph monitors & managers] *****************************************
2026-04-08 03:08:08.915943 | instance |
2026-04-08 03:08:08.916265 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 03:08:08.916627 | instance | Wednesday 08 April 2026  03:08:08 +0000 (0:00:00.178)       0:00:01.644 *******
2026-04-08 03:08:09.944978 | instance | ok: [instance]
2026-04-08 03:08:09.945234 | instance |
2026-04-08 03:08:09.945509 | instance | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-04-08 03:08:09.945843 | instance | Wednesday 08 April 2026  03:08:09 +0000 (0:00:01.029)       0:00:02.674 *******
2026-04-08 03:08:10.345983 | instance | ok: [instance]
2026-04-08 03:08:10.346203 | instance |
2026-04-08 03:08:10.346475 | instance | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-04-08 03:08:10.346741 | instance | Wednesday 08 April 2026  03:08:10 +0000 (0:00:00.400)       0:00:03.075 *******
2026-04-08 03:08:10.392909 | instance | skipping: [instance]
2026-04-08 03:08:10.393093 | instance |
2026-04-08 03:08:10.393355 | instance | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] ***
2026-04-08 03:08:10.393619 | instance | Wednesday 08 April 2026  03:08:10 +0000 (0:00:00.047)       0:00:03.122 *******
2026-04-08 03:08:10.793709 | instance | changed: [instance]
2026-04-08 03:08:10.793807 | instance |
2026-04-08 03:08:10.793989 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-04-08 03:08:10.794156 | instance | Wednesday 08 April 2026  03:08:10 +0000 (0:00:00.400)       0:00:03.522 *******
2026-04-08 03:08:10.873188 | instance | ok: [instance] => {
2026-04-08 03:08:10.873404 | instance |     "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.0/runc.amd64"
2026-04-08 03:08:10.873655 | instance | }
2026-04-08 03:08:10.873897 | instance |
2026-04-08 03:08:10.874161 | instance | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-04-08 03:08:10.874428 | instance | Wednesday 08 April 2026  03:08:10 +0000 (0:00:00.080)       0:00:03.602 *******
2026-04-08 03:08:11.719072 | instance | changed: [instance]
2026-04-08 03:08:11.719298 | instance |
2026-04-08 03:08:11.719581 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-04-08 03:08:11.719856 | instance | Wednesday 08 April 2026  03:08:11 +0000 (0:00:00.845)       0:00:04.448 *******
2026-04-08 03:08:11.772298 | instance | skipping: [instance]
2026-04-08 03:08:11.772655 | instance |
2026-04-08 03:08:11.773026 | instance | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-04-08 03:08:11.773411 | instance | Wednesday 08 April 2026  03:08:11 +0000 (0:00:00.053)       0:00:04.501 *******
2026-04-08 03:08:11.827786 | instance | skipping: [instance]
2026-04-08 03:08:11.827992 | instance |
2026-04-08 03:08:11.828257 | instance | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-04-08 03:08:11.828574 | instance | Wednesday 08 April 2026  03:08:11 +0000 (0:00:00.055)       0:00:04.557 *******
2026-04-08 03:08:12.150359 | instance | ok: [instance]
2026-04-08 03:08:12.150551 | instance |
2026-04-08 03:08:12.150840 | instance | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-04-08 03:08:12.151137 | instance | Wednesday 08 April 2026  03:08:12 +0000 (0:00:00.322)       0:00:04.879 *******
2026-04-08 03:08:13.511937 | instance | ok: [instance]
2026-04-08 03:08:13.512130 | instance |
2026-04-08 03:08:13.512455 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-04-08 03:08:13.512725 | instance | Wednesday 08 April 2026  03:08:13 +0000 (0:00:01.361)       0:00:06.241 *******
2026-04-08 03:08:13.574487 | instance | ok: [instance] => {
2026-04-08 03:08:13.574654 | instance |     "msg": "https://github.com/containerd/containerd/releases/download/v2.2.0/containerd-2.2.0-linux-amd64.tar.gz"
2026-04-08 03:08:13.574766 | instance | }
2026-04-08 03:08:13.574883 | instance |
2026-04-08 03:08:13.575004 | instance | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-04-08 03:08:13.575116 | instance | Wednesday 08 April 2026  03:08:13 +0000 (0:00:00.062)       0:00:06.304 *******
2026-04-08 03:08:14.393719 | instance | changed: [instance]
2026-04-08 03:08:14.393918 | instance |
2026-04-08 03:08:14.394192 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-04-08 03:08:14.394498 | instance | Wednesday 08 April 2026  03:08:14 +0000 (0:00:00.819)       0:00:07.123 *******
2026-04-08 03:08:17.400756 | instance | changed: [instance]
2026-04-08 03:08:17.401011 | instance |
2026-04-08 03:08:17.401431 | instance | TASK [vexxhost.containers.containerd : Install SELinux packages] ***************
2026-04-08 03:08:17.402131 | instance | Wednesday 08 April 2026  03:08:17 +0000 (0:00:03.006)       0:00:10.130 *******
2026-04-08 03:08:17.435320 | instance | skipping: [instance]
2026-04-08 03:08:17.435597 | instance |
2026-04-08 03:08:17.435864 | instance | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] ***
2026-04-08 03:08:17.436130 | instance | Wednesday 08 April 2026  03:08:17 +0000 (0:00:00.034)       0:00:10.164 *******
2026-04-08 03:08:17.473351 | instance | skipping: [instance]
2026-04-08 03:08:17.473583 | instance |
2026-04-08 03:08:17.473873 | instance | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ********
2026-04-08 03:08:17.474132 | instance | Wednesday 08 April 2026  03:08:17 +0000 (0:00:00.038)       0:00:10.203 *******
2026-04-08 03:08:17.507783 | instance | skipping: [instance]
2026-04-08 03:08:17.508013 | instance |
2026-04-08 03:08:17.508304 | instance | TASK [vexxhost.containers.containerd : Install AppArmor packages] **************
2026-04-08 03:08:17.508579 | instance | Wednesday 08 April 2026  03:08:17 +0000 (0:00:00.034)       0:00:10.237 *******
2026-04-08 03:08:22.895439 | instance | changed: [instance]
2026-04-08 03:08:22.895535 | instance |
2026-04-08 03:08:22.895542 | instance | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] ***
2026-04-08 03:08:22.895548 | instance | Wednesday 08 April 2026  03:08:22 +0000 (0:00:05.384)       0:00:15.622 *******
2026-04-08 03:08:23.636005 | instance | changed: [instance]
2026-04-08 03:08:23.636150 | instance |
2026-04-08 03:08:23.636597 | instance | TASK [vexxhost.containers.containerd : Create folders for configuration] *******
2026-04-08 03:08:23.636911 | instance | Wednesday 08 April 2026  03:08:23 +0000 (0:00:00.742)       0:00:16.365 *******
2026-04-08 03:08:25.080134 | instance | changed: [instance] => (item={'path': '/etc/containerd'})
2026-04-08 03:08:25.080231 | instance | changed: [instance] => (item={'path': '/var/lib/containerd', 'mode': '0o700'})
2026-04-08 03:08:25.080344 | instance | changed: [instance] => (item={'path': '/run/containerd', 'mode': '0o711'})
2026-04-08 03:08:25.081094 | instance | changed: [instance] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'})
2026-04-08 03:08:25.081152 | instance | changed: [instance] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'})
2026-04-08 03:08:25.081159 | instance |
2026-04-08 03:08:25.081164 | instance | TASK [vexxhost.containers.containerd : Create containerd config file] **********
2026-04-08 03:08:25.081169 | instance | Wednesday 08 April 2026  03:08:25 +0000 (0:00:01.444)       0:00:17.809 *******
2026-04-08 03:08:25.757500 | instance | changed: [instance]
2026-04-08 03:08:25.757586 | instance |
2026-04-08 03:08:25.757654 | instance | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ********
2026-04-08 03:08:25.758171 | instance | Wednesday 08 April 2026  03:08:25 +0000 (0:00:00.668)       0:00:18.478 *******
2026-04-08 03:08:25.758211 | instance |
2026-04-08 03:08:25.758217 | instance | RUNNING HANDLER [vexxhost.containers.containerd : Reload systemd] **************
2026-04-08 03:08:25.758221 | instance | Wednesday 08 April 2026  03:08:25 +0000 (0:00:00.008)       0:00:18.487 *******
2026-04-08 03:08:26.866871 | instance | ok: [instance]
2026-04-08 03:08:26.866954 | instance |
2026-04-08 03:08:26.867237 | instance | RUNNING HANDLER [vexxhost.containers.containerd : Restart containerd] **********
2026-04-08 03:08:26.867272 | instance | Wednesday 08 April 2026  03:08:26 +0000 (0:00:01.109)       0:00:19.596 *******
2026-04-08 03:08:27.448115 | instance | changed: [instance]
2026-04-08 03:08:27.448198 | instance |
2026-04-08 03:08:27.448417 | instance | TASK [vexxhost.containers.containerd : Enable and start service] ***************
2026-04-08 03:08:27.448542 | instance | Wednesday 08 April 2026  03:08:27 +0000 (0:00:00.580)       0:00:20.177 *******
2026-04-08 03:08:28.149640 | instance | changed: [instance]
2026-04-08 03:08:28.149800 | instance |
2026-04-08 03:08:28.149812 | instance | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-04-08 03:08:28.149883 | instance | Wednesday 08 April 2026  03:08:28 +0000 (0:00:00.701)       0:00:20.879 *******
2026-04-08 03:08:28.487336 | instance | ok: [instance]
2026-04-08 03:08:28.487452 | instance |
2026-04-08 03:08:28.487465 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-04-08 03:08:28.487669 | instance | Wednesday 08 April 2026  03:08:28 +0000 (0:00:00.337)       0:00:21.216 *******
2026-04-08 03:08:28.551003 | instance | ok: [instance] => {
2026-04-08 03:08:28.551639 | instance |     "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz"
2026-04-08 03:08:28.551705 | instance | }
2026-04-08 03:08:28.551712 | instance |
2026-04-08 03:08:28.551716 | instance | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-04-08 03:08:28.551722 | instance | Wednesday 08 April 2026  03:08:28 +0000 (0:00:00.063)       0:00:21.280 *******
2026-04-08 03:08:29.620555 | instance | changed: [instance]
2026-04-08 03:08:29.620678 | instance |
2026-04-08 03:08:29.620697 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-04-08 03:08:29.620893 | instance | Wednesday 08 April 2026  03:08:29 +0000 (0:00:01.069)       0:00:22.350 *******
2026-04-08 03:08:34.050005 | instance | changed: [instance]
2026-04-08 03:08:34.050077 | instance |
2026-04-08 03:08:34.050358 | instance | TASK [vexxhost.containers.docker : Install AppArmor packages] ******************
2026-04-08 03:08:34.050402 | instance | Wednesday 08 April 2026  03:08:34 +0000 (0:00:04.429)       0:00:26.780 *******
2026-04-08 03:08:35.168303 | instance | ok: [instance]
2026-04-08 03:08:35.168418 | instance |
2026-04-08 03:08:35.168614 | instance | TASK [vexxhost.containers.docker : Ensure group "docker" exists] ***************
2026-04-08 03:08:35.169082 | instance | Wednesday 08 April 2026  03:08:35 +0000 (0:00:01.117)       0:00:27.897 *******
2026-04-08 03:08:35.678466 | instance | changed: [instance]
2026-04-08 03:08:35.678548 | instance |
2026-04-08 03:08:35.678829 | instance | TASK [vexxhost.containers.docker : Create systemd service file for docker] *****
2026-04-08 03:08:35.678902 | instance | Wednesday 08 April 2026  03:08:35 +0000 (0:00:00.509)       0:00:28.407 *******
2026-04-08 03:08:36.256843 | instance | changed: [instance]
2026-04-08 03:08:36.256921 | instance |
2026-04-08 03:08:36.257202 | instance | TASK [vexxhost.containers.docker : Create folders for configuration] ***********
2026-04-08 03:08:36.257249 | instance | Wednesday 08 April 2026  03:08:36 +0000 (0:00:00.578)       0:00:28.986 *******
2026-04-08 03:08:37.164160 | instance | changed: [instance] => (item={'path': '/etc/docker'})
2026-04-08 03:08:37.164232 | instance | changed: [instance] => (item={'path': '/var/lib/docker', 'mode': '0o710'})
2026-04-08 03:08:37.164503 | instance | changed: [instance] => (item={'path': '/run/docker', 'mode': '0o711'})
2026-04-08 03:08:37.164547 | instance |
2026-04-08 03:08:37.164774 | instance | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ******
2026-04-08 03:08:37.164818 | instance | Wednesday 08 April 2026  03:08:37 +0000 (0:00:00.907)       0:00:29.893 *******
2026-04-08 03:08:37.758194 | instance | changed: [instance]
2026-04-08 03:08:37.758300 | instance |
2026-04-08 03:08:37.758575 | instance | TASK [vexxhost.containers.docker : Create docker daemon config file] ***********
2026-04-08 03:08:37.758595 | instance | Wednesday 08 April 2026  03:08:37 +0000 (0:00:00.594)       0:00:30.488 *******
2026-04-08 03:08:38.360505 | instance | changed: [instance]
2026-04-08 03:08:38.360897 | instance |
2026-04-08 03:08:38.360918 | instance | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************
2026-04-08 03:08:38.360934 | instance | Wednesday 08 April 2026  03:08:38 +0000 (0:00:00.594)       0:00:31.082 *******
2026-04-08 03:08:38.360951 | instance |
2026-04-08 03:08:38.361195 | instance | RUNNING HANDLER [vexxhost.containers.containerd : Reload systemd] **************
2026-04-08 03:08:38.361234 | instance | Wednesday 08 April 2026  03:08:38 +0000 (0:00:00.008)       0:00:31.090 *******
2026-04-08 03:08:39.192990 | instance | ok: [instance]
2026-04-08 03:08:39.193070 | instance |
2026-04-08 03:08:39.193370 | instance | RUNNING HANDLER [vexxhost.containers.docker : Restart docker] ******************
2026-04-08 03:08:39.193413 | instance | Wednesday 08 April 2026  03:08:39 +0000 (0:00:00.832)       0:00:31.922 *******
2026-04-08 03:08:40.384442 | instance | changed: [instance]
2026-04-08 03:08:40.384533 | instance |
2026-04-08 03:08:40.384794 | instance | TASK [vexxhost.containers.docker : Enable and start service] *******************
2026-04-08 03:08:40.384858 | instance | Wednesday 08 April 2026  03:08:40 +0000 (0:00:01.191)       0:00:33.113 *******
2026-04-08 03:08:41.068942 | instance | changed: [instance]
2026-04-08 03:08:41.069064 | instance |
2026-04-08 03:08:41.069081 | instance | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ******
2026-04-08 03:08:41.069291 | instance | Wednesday 08 April 2026  03:08:41 +0000 (0:00:00.684)       0:00:33.798 *******
2026-04-08 03:08:41.133137 | instance | ok: [instance] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/ubuntu-22.04.yml)
2026-04-08 03:08:41.133231 | instance |
2026-04-08 03:08:41.133511 | instance | TASK [vexxhost.ceph.cephadm : Install packages] ********************************
2026-04-08 03:08:41.133557 | instance | Wednesday 08 April 2026  03:08:41 +0000 (0:00:00.064)       0:00:33.862 *******
2026-04-08 03:08:46.545425 | instance | changed: [instance]
2026-04-08 03:08:46.545484 | instance |
2026-04-08 03:08:46.545490 | instance | TASK [vexxhost.ceph.cephadm : Ensure services are started] *********************
2026-04-08 03:08:46.545495 | instance | Wednesday 08 April 2026  03:08:46 +0000 (0:00:05.410)       0:00:39.272 *******
2026-04-08 03:08:47.421996 | instance | ok: [instance] => (item=chronyd)
2026-04-08 03:08:47.422758 | instance | ok: [instance] => (item=sshd)
2026-04-08 03:08:47.422809 | instance |
2026-04-08 03:08:47.422818 | instance | TASK [vexxhost.ceph.cephadm : Download "cephadm"] ******************************
2026-04-08 03:08:47.422826 | instance | Wednesday 08 April 2026  03:08:47 +0000 (0:00:00.878)       0:00:40.151 *******
2026-04-08 03:08:47.841079 | instance | changed: [instance]
2026-04-08 03:08:47.841144 | instance |
2026-04-08 03:08:47.841420 | instance | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ********************
2026-04-08 03:08:47.841462 | instance | Wednesday 08 April 2026  03:08:47 +0000 (0:00:00.419)       0:00:40.570 *******
2026-04-08 03:08:48.173621 | instance | ok: [instance]
2026-04-08 03:08:48.173699 | instance |
2026-04-08 03:08:48.173950 | instance | TASK [vexxhost.ceph.cephadm : Ensure "cephadm" user is present] ****************
2026-04-08 03:08:48.173988 | instance | Wednesday 08 April 2026  03:08:48 +0000 (0:00:00.332)       0:00:40.903 *******
2026-04-08 03:08:48.886696 | instance | changed: [instance]
2026-04-08 03:08:48.886760 | instance |
2026-04-08 03:08:48.887051 | instance | TASK [vexxhost.ceph.cephadm : Allow "cephadm" user to have passwordless sudo] ***
2026-04-08 03:08:48.887087 | instance | Wednesday 08 April 2026  03:08:48 +0000 (0:00:00.713)       0:00:41.616 *******
2026-04-08 03:08:49.438446 | instance | changed: [instance]
2026-04-08 03:08:49.438817 | instance |
2026-04-08 03:08:49.438857 | instance | TASK [vexxhost.ceph.mon : Get `cephadm ls` status] *****************************
2026-04-08 03:08:49.438862 | instance | Wednesday 08 April 2026  03:08:49 +0000 (0:00:00.551)       0:00:42.168 *******
2026-04-08 03:08:51.249491 | instance | ok: [instance]
2026-04-08 03:08:51.249572 | instance |
2026-04-08 03:08:51.249782 | instance | TASK [vexxhost.ceph.mon : Parse the `cephadm ls` output] ***********************
2026-04-08 03:08:51.249826 | instance | Wednesday 08 April 2026  03:08:51 +0000 (0:00:01.811)       0:00:43.979 *******
2026-04-08 03:08:51.306582 | instance | ok: [instance]
2026-04-08 03:08:51.307420 | instance |
2026-04-08 03:08:51.307477 | instance | TASK [vexxhost.ceph.mon : Assimilate existing configs in `ceph.conf`] **********
2026-04-08 03:08:51.307489 | instance | Wednesday 08 April 2026  03:08:51 +0000 (0:00:00.056)       0:00:44.035 *******
2026-04-08 03:08:51.350456 | instance | skipping: [instance]
2026-04-08 03:08:51.350537 | instance |
2026-04-08 03:08:51.350882 | instance | TASK [vexxhost.ceph.mon : Adopt monitor to cluster] ****************************
2026-04-08 03:08:51.350931 | instance | Wednesday 08 April 2026  03:08:51 +0000 (0:00:00.043)       0:00:44.079 *******
2026-04-08 03:08:51.394083 | instance | skipping: [instance]
2026-04-08 03:08:51.394155 | instance |
2026-04-08 03:08:51.394428 | instance | TASK [vexxhost.ceph.mon : Adopt manager to cluster] ****************************
2026-04-08 03:08:51.394470 | instance | Wednesday 08 April 2026  03:08:51 +0000 (0:00:00.043)       0:00:44.123 *******
2026-04-08 03:08:51.432487 | instance | skipping: [instance]
2026-04-08 03:08:51.432556 | instance |
2026-04-08 03:08:51.432792 | instance | TASK [vexxhost.ceph.mon : Enable "cephadm" mgr module] *************************
2026-04-08 03:08:51.432834 | instance | Wednesday 08 April 2026  03:08:51 +0000 (0:00:00.038)       0:00:44.162 *******
2026-04-08 03:08:51.478724 | instance | skipping: [instance]
2026-04-08 03:08:51.478873 | instance |
2026-04-08 03:08:51.479093 | instance | TASK [vexxhost.ceph.mon : Set orchestrator backend to "cephadm"] ***************
2026-04-08 03:08:51.479251 | instance | Wednesday 08 April 2026  03:08:51 +0000 (0:00:00.046)       0:00:44.208 *******
2026-04-08 03:08:51.522031 | instance | skipping: [instance]
2026-04-08 03:08:51.522122 | instance |
2026-04-08 03:08:51.522257 | instance | TASK [vexxhost.ceph.mon : Use `cephadm` user for cephadm] **********************
2026-04-08 03:08:51.522442 | instance | Wednesday 08 April 2026  03:08:51 +0000 (0:00:00.043)       0:00:44.251 *******
2026-04-08 03:08:51.566990 | instance | skipping: [instance]
2026-04-08 03:08:51.567135 | instance |
2026-04-08 03:08:51.567336 | instance | TASK [vexxhost.ceph.mon : Generate "cephadm" key] ******************************
2026-04-08 03:08:51.567492 | instance | Wednesday 08 April 2026  03:08:51 +0000 (0:00:00.045)       0:00:44.296 *******
2026-04-08 03:08:51.604841 | instance | skipping: [instance]
2026-04-08 03:08:51.604941 | instance |
2026-04-08 03:08:51.605119 | instance | TASK [vexxhost.ceph.mon : Set Ceph Monitor IP address] *************************
2026-04-08 03:08:51.605272 | instance | Wednesday 08 April 2026  03:08:51 +0000 (0:00:00.037)       0:00:44.333 *******
2026-04-08 03:08:51.728936 | instance | ok: [instance]
2026-04-08 03:08:51.729046 | instance |
2026-04-08 03:08:51.729178 | instance | TASK [vexxhost.ceph.mon : Check if any node is bootstrapped] *******************
2026-04-08 03:08:51.729364 | instance | Wednesday 08 April 2026  03:08:51 +0000 (0:00:00.124)       0:00:44.458 *******
2026-04-08 03:08:52.063082 | instance | ok: [instance] => (item=instance)
2026-04-08 03:08:52.063201 | instance |
2026-04-08 03:08:52.063217 | instance | TASK [vexxhost.ceph.mon : Select pre-existing bootstrap node if exists] ********
2026-04-08 03:08:52.063400 | instance | Wednesday 08 April 2026  03:08:52 +0000 (0:00:00.334)       0:00:44.792 *******
2026-04-08 03:08:52.120970 | instance | ok: [instance]
2026-04-08 03:08:52.121032 | instance |
2026-04-08 03:08:52.121039 | instance | TASK [vexxhost.ceph.mon : Bootstrap cluster] ***********************************
2026-04-08 03:08:52.121045 | instance | Wednesday 08 April 2026  03:08:52 +0000 (0:00:00.057)       0:00:44.850 *******
2026-04-08 03:08:52.198178 | instance | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/mon/tasks/bootstrap-ceph.yml for instance
2026-04-08 03:08:52.198289 | instance |
2026-04-08 03:08:52.198593 | instance | TASK [vexxhost.ceph.mon : Generate temporary file for "ceph.conf"] *************
2026-04-08 03:08:52.198642 | instance | Wednesday 08 April 2026  03:08:52 +0000 (0:00:00.077)       0:00:44.927 *******
2026-04-08 03:08:52.686637 | instance | changed: [instance]
2026-04-08 03:08:52.686723 | instance |
2026-04-08 03:08:52.686988 | instance | TASK [vexxhost.ceph.mon : Include extra configuration values] ******************
2026-04-08 03:08:52.687054 | instance | Wednesday 08 April 2026  03:08:52 +0000 (0:00:00.488)       0:00:45.416 *******
2026-04-08 03:08:53.774614 | instance | changed: [instance] => (item={'option': 'mon allow pool size one', 'section': 'global', 'value': True})
2026-04-08 03:08:53.775313 | instance | changed: [instance] => (item={'option': 'osd crush chooseleaf type', 'section': 'global', 'value': 0})
2026-04-08 03:08:53.775367 | instance | changed: [instance] => (item={'option': 'auth allow insecure global id reclaim', 'section': 'mon', 'value': False})
2026-04-08 03:08:53.775375 | instance |
2026-04-08 03:08:53.775380 | instance | TASK [vexxhost.ceph.mon : Run Bootstrap coomand] *******************************
2026-04-08 03:08:53.775385 | instance | Wednesday 08 April 2026  03:08:53 +0000 (0:00:01.088)       0:00:46.504 *******
2026-04-08 03:10:59.967233 | instance | ok: [instance]
2026-04-08 03:10:59.967322 | instance |
2026-04-08 03:10:59.967335 | instance | TASK [vexxhost.ceph.mon : Remove temporary file for "ceph.conf"] ***************
2026-04-08 03:10:59.967345 | instance | Wednesday 08 April 2026  03:10:59 +0000 (0:02:06.189)       0:02:52.693 *******
2026-04-08 03:11:00.281722 | instance | changed: [instance]
2026-04-08 03:11:00.281796 | instance |
2026-04-08 03:11:00.282074 | instance | TASK [vexxhost.ceph.mon : Set bootstrap node] **********************************
2026-04-08 03:11:00.282112 | instance | Wednesday 08 April 2026  03:11:00 +0000 (0:00:00.317)       0:02:53.011 *******
2026-04-08 03:11:00.324890 | instance | ok: [instance]
2026-04-08 03:11:00.325274 | instance |
2026-04-08 03:11:00.325318 | instance | TASK [Install Ceph host] *******************************************************
2026-04-08 03:11:00.325325 | instance | Wednesday 08 April 2026  03:11:00 +0000 (0:00:00.042)       0:02:53.054 *******
2026-04-08 03:11:00.409617 | instance | included: vexxhost.ceph.cephadm_host for instance
2026-04-08 03:11:00.409706 | instance |
2026-04-08 03:11:00.409780 | instance | TASK [vexxhost.ceph.cephadm_host : Get public SSH key for "cephadm" user] ******
2026-04-08 03:11:00.409916 | instance | Wednesday 08 April 2026  03:11:00 +0000 (0:00:00.085)       0:02:53.139 *******
2026-04-08 03:11:02.089752 | instance | ok: [instance]
2026-04-08 03:11:02.089825 | instance |
2026-04-08 03:11:02.090161 | instance | TASK [vexxhost.ceph.cephadm_host : Set fact with public SSH key for "cephadm" user] ***
2026-04-08 03:11:02.090200 | instance | Wednesday 08 April 2026  03:11:02 +0000 (0:00:01.680)       0:02:54.819 *******
2026-04-08 03:11:02.149247 | instance | ok: [instance] => (item=instance)
2026-04-08 03:11:02.149318 | instance |
2026-04-08 03:11:02.149601 | instance | TASK [vexxhost.ceph.cephadm_host : Set authorized key for "cephadm"] ***********
2026-04-08 03:11:02.149644 | instance | Wednesday 08 April 2026  03:11:02 +0000 (0:00:00.059)       0:02:54.879 *******
2026-04-08 03:11:02.678339 | instance | ok: [instance]
2026-04-08 03:11:02.678457 | instance |
2026-04-08 03:11:02.678527 | instance | TASK [vexxhost.ceph.cephadm_host : Add new host to Ceph] ***********************
2026-04-08 03:11:02.678684 | instance | Wednesday 08 April 2026  03:11:02 +0000 (0:00:00.529)       0:02:55.408 *******
2026-04-08 03:11:04.785486 | instance | ok: [instance]
2026-04-08 03:11:04.785868 | instance |
2026-04-08 03:11:04.785882 | instance | TASK [vexxhost.ceph.mon : Configure "mon" label for monitors] ******************
2026-04-08 03:11:04.785892 | instance | Wednesday 08 April 2026  03:11:04 +0000 (0:00:02.107)       0:02:57.515 *******
2026-04-08 03:11:06.583882 | instance | ok: [instance]
2026-04-08 03:11:06.583954 | instance |
2026-04-08 03:11:06.584232 | instance | TASK [vexxhost.ceph.mon : Validate monitor exist] ******************************
2026-04-08 03:11:06.584276 | instance | Wednesday 08 April 2026  03:11:06 +0000 (0:00:01.798)       0:02:59.313 *******
2026-04-08 03:11:16.949583 | instance | ok: [instance]
2026-04-08 03:11:16.949653 | instance |
2026-04-08 03:11:16.949941 | instance | TASK [Install Ceph host] *******************************************************
2026-04-08 03:11:16.949980 | instance | Wednesday 08 April 2026  03:11:16 +0000 (0:00:10.365)       0:03:09.679 *******
2026-04-08 03:11:17.023152 | instance | included: vexxhost.ceph.cephadm_host for instance
2026-04-08 03:11:17.023533 | instance |
2026-04-08 03:11:17.023575 | instance | TASK [vexxhost.ceph.cephadm_host : Get public SSH key for "cephadm" user] ******
2026-04-08 03:11:17.023581 | instance | Wednesday 08 April 2026  03:11:17 +0000 (0:00:00.073)       0:03:09.753 *******
2026-04-08 03:11:17.069710 | instance | skipping: [instance]
2026-04-08 03:11:17.070129 | instance |
2026-04-08 03:11:17.070174 | instance | TASK [vexxhost.ceph.cephadm_host : Set fact with public SSH key for "cephadm" user] ***
2026-04-08 03:11:17.070181 | instance | Wednesday 08 April 2026  03:11:17 +0000 (0:00:00.046)       0:03:09.799 *******
2026-04-08 03:11:17.118915 | instance | skipping: [instance] => (item=instance)
2026-04-08 03:11:17.119378 | instance | skipping: [instance]
2026-04-08 03:11:17.119400 | instance |
2026-04-08 03:11:17.119405 | instance | TASK [vexxhost.ceph.cephadm_host : Set authorized key for "cephadm"] ***********
2026-04-08 03:11:17.119410 | instance | Wednesday 08 April 2026  03:11:17 +0000 (0:00:00.049)       0:03:09.848 *******
2026-04-08 03:11:17.490202 | instance | ok: [instance]
2026-04-08 03:11:17.490621 | instance |
2026-04-08 03:11:17.490661 | instance | TASK [vexxhost.ceph.cephadm_host : Add new host to Ceph] ***********************
2026-04-08 03:11:17.490667 | instance | Wednesday 08 April 2026  03:11:17 +0000 (0:00:00.371)       0:03:10.219 *******
2026-04-08 03:11:19.518188 | instance | ok: [instance]
2026-04-08 03:11:19.518258 | instance |
2026-04-08 03:11:19.518481 | instance | TASK [vexxhost.ceph.mgr : Configure "mgr" label for managers] ******************
2026-04-08 03:11:19.518512 | instance | Wednesday 08 April 2026  03:11:19 +0000 (0:00:02.028)       0:03:12.248 *******
2026-04-08 03:11:21.271409 | instance | ok: [instance]
2026-04-08 03:11:21.271487 | instance |
2026-04-08 03:11:21.271757 | instance | TASK [vexxhost.ceph.mgr : Validate manager exist] ******************************
2026-04-08 03:11:21.271796 | instance | Wednesday 08 April 2026  03:11:21 +0000 (0:00:01.753)       0:03:14.001 *******
2026-04-08 03:11:22.910032 | instance | ok: [instance]
2026-04-08 03:11:22.910111 | instance |
2026-04-08 03:11:22.910337 | instance | TASK [vexxhost.ceph.mgr : Enable the Ceph Manager prometheus module] ***********
2026-04-08 03:11:22.910376 | instance | Wednesday 08 April 2026  03:11:22 +0000 (0:00:01.638)       0:03:15.640 *******
2026-04-08 03:11:24.831626 | instance | ok: [instance]
2026-04-08 03:11:24.831717 | instance |
2026-04-08 03:11:24.831799 | instance | PLAY [Deploy Ceph OSDs] ********************************************************
2026-04-08 03:11:24.831908 | instance |
2026-04-08 03:11:24.832026 | instance | TASK [Gathering Facts] *********************************************************
2026-04-08 03:11:24.832146 | instance | Wednesday 08 April 2026  03:11:24 +0000 (0:00:01.921)       0:03:17.561 *******
2026-04-08 03:11:25.860453 | instance | ok: [instance]
2026-04-08 03:11:25.860534 | instance |
2026-04-08 03:11:25.860585 | instance | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-04-08 03:11:25.860714 | instance | Wednesday 08 April 2026  03:11:25 +0000 (0:00:01.028)       0:03:18.590 *******
2026-04-08 03:11:26.174834 | instance | ok: [instance]
2026-04-08 03:11:26.174905 | instance |
2026-04-08 03:11:26.175033 | instance | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-04-08 03:11:26.175154 | instance | Wednesday 08 April 2026  03:11:26 +0000 (0:00:00.314)       0:03:18.904 *******
2026-04-08 03:11:26.210875 | instance | skipping: [instance]
2026-04-08 03:11:26.210948 | instance |
2026-04-08 03:11:26.211083 | instance | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] ***
2026-04-08 03:11:26.211197 | instance | Wednesday 08 April 2026  03:11:26 +0000 (0:00:00.036)       0:03:18.940 *******
2026-04-08 03:11:26.541558 | instance | ok: [instance]
2026-04-08 03:11:26.541637 | instance |
2026-04-08 03:11:26.541757 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-04-08 03:11:26.541883 | instance | Wednesday 08 April 2026  03:11:26 +0000 (0:00:00.330)       0:03:19.270 *******
2026-04-08 03:11:26.601613 | instance | ok: [instance] => {
2026-04-08 03:11:26.601693 | instance |     "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.0/runc.amd64"
2026-04-08 03:11:26.601796 | instance | }
2026-04-08 03:11:26.601897 | instance |
2026-04-08 03:11:26.602021 | instance | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-04-08 03:11:26.602145 | instance | Wednesday 08 April 2026  03:11:26 +0000 (0:00:00.060)       0:03:19.331 *******
2026-04-08 03:11:27.012755 | instance | ok: [instance]
2026-04-08 03:11:27.012838 | instance |
2026-04-08 03:11:27.012940 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-04-08 03:11:27.013061 | instance | Wednesday 08 April 2026  03:11:27 +0000 (0:00:00.410)       0:03:19.742 *******
2026-04-08 03:11:27.066441 | instance | skipping: [instance]
2026-04-08 03:11:27.066548 | instance |
2026-04-08 03:11:27.066680 | instance | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-04-08 03:11:27.066807 | instance | Wednesday 08 April 2026  03:11:27 +0000 (0:00:00.053)       0:03:19.795 *******
2026-04-08 03:11:27.115458 | instance | skipping: [instance]
2026-04-08 03:11:27.115535 | instance |
2026-04-08 03:11:27.115678 | instance | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-04-08 03:11:27.115838 | instance | Wednesday 08 April 2026  03:11:27 +0000 (0:00:00.049)       0:03:19.845 *******
2026-04-08 03:11:27.429354 | instance | ok: [instance]
2026-04-08 03:11:27.429440 | instance |
2026-04-08 03:11:27.429553 | instance | TASK [vexxhost.containers.package : Update state for tar] **********************
2026-04-08 03:11:27.429672 | instance | Wednesday 08 April 2026  03:11:27 +0000 (0:00:00.313)       0:03:20.159 *******
2026-04-08 03:11:28.814607 | instance | ok: [instance]
2026-04-08 03:11:28.814683 | instance |
2026-04-08 03:11:28.814809 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-04-08 03:11:28.814923 | instance | Wednesday 08 April 2026  03:11:28 +0000 (0:00:01.385)       0:03:21.544 *******
2026-04-08 03:11:28.878575 | instance | ok: [instance] => {
2026-04-08 03:11:28.878756 | instance |     "msg": "https://github.com/containerd/containerd/releases/download/v2.2.0/containerd-2.2.0-linux-amd64.tar.gz"
2026-04-08 03:11:28.878917 | instance | }
2026-04-08 03:11:28.879079 | instance |
2026-04-08 03:11:28.879262 | instance | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-04-08 03:11:28.879426 | instance | Wednesday 08 April 2026  03:11:28 +0000 (0:00:00.063)       0:03:21.608 *******
2026-04-08 03:11:29.343961 | instance | ok: [instance]
2026-04-08 03:11:29.344083 | instance |
2026-04-08 03:11:29.344323 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-04-08 03:11:29.344559 | instance | Wednesday 08 April 2026  03:11:29 +0000 (0:00:00.465)       0:03:22.073 *******
2026-04-08 03:11:31.384374 | instance | ok: [instance]
2026-04-08 03:11:31.384489 | instance |
2026-04-08 03:11:31.384623 | instance | TASK [vexxhost.containers.containerd : Install SELinux packages] ***************
2026-04-08 03:11:31.384744 | instance | Wednesday 08 April 2026  03:11:31 +0000 (0:00:02.040)       0:03:24.114 *******
2026-04-08 03:11:31.416464 | instance | skipping: [instance]
2026-04-08 03:11:31.416599 | instance |
2026-04-08 03:11:31.416771 | instance | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] ***
2026-04-08 03:11:31.416937 | instance | Wednesday 08 April 2026  03:11:31 +0000 (0:00:00.032)       0:03:24.146 *******
2026-04-08 03:11:31.443191 | instance | skipping: [instance]
2026-04-08 03:11:31.443331 | instance |
2026-04-08 03:11:31.443507 | instance | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ********
2026-04-08 03:11:31.443678 | instance | Wednesday 08 April 2026  03:11:31 +0000 (0:00:00.026)       0:03:24.172 *******
2026-04-08 03:11:31.476268 | instance | skipping: [instance]
2026-04-08 03:11:31.476481 | instance |
2026-04-08 03:11:31.476643 | instance | TASK [vexxhost.containers.containerd : Install AppArmor packages] **************
2026-04-08 03:11:31.476822 | instance | Wednesday 08 April 2026  03:11:31 +0000 (0:00:00.033)       0:03:24.206 *******
2026-04-08 03:11:32.869098 | instance | ok: [instance]
2026-04-08 03:11:32.869218 | instance |
2026-04-08 03:11:32.869399 | instance | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] ***
2026-04-08 03:11:32.869571 | instance | Wednesday 08 April 2026  03:11:32 +0000 (0:00:01.392)       0:03:25.598 *******
2026-04-08 03:11:33.434072 | instance | ok: [instance]
2026-04-08 03:11:33.434201 | instance |
2026-04-08 03:11:33.434384 | instance | TASK [vexxhost.containers.containerd : Create folders for configuration] *******
2026-04-08 03:11:33.434638 | instance | Wednesday 08 April 2026  03:11:33 +0000 (0:00:00.565)       0:03:26.163 *******
2026-04-08 03:11:34.906923 | instance | ok: [instance] => (item={'path': '/etc/containerd'})
2026-04-08 03:11:34.906988 | instance | ok: [instance] => (item={'path': '/var/lib/containerd', 'mode': '0o700'})
2026-04-08 03:11:34.907137 | instance | ok: [instance] => (item={'path': '/run/containerd', 'mode': '0o711'})
2026-04-08 03:11:34.907301 | instance | ok: [instance] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'})
2026-04-08 03:11:34.907433 | instance | ok: [instance] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'})
2026-04-08 03:11:34.907553 | instance |
2026-04-08 03:11:34.907692 | instance | TASK [vexxhost.containers.containerd : Create containerd config file] **********
2026-04-08 03:11:34.907802 | instance | Wednesday 08 April 2026  03:11:34 +0000 (0:00:01.473)       0:03:27.636 *******
2026-04-08 03:11:35.496816 | instance | ok: [instance]
2026-04-08 03:11:35.496891 | instance |
2026-04-08 03:11:35.497018 | instance | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ********
2026-04-08 03:11:35.497151 | instance | Wednesday 08 April 2026  03:11:35 +0000 (0:00:00.583)       0:03:28.220 *******
2026-04-08 03:11:35.497268 | instance |
2026-04-08 03:11:35.497395 | instance | TASK [vexxhost.containers.containerd : Enable and start service] ***************
2026-04-08 03:11:35.497523 | instance | Wednesday 08 April 2026  03:11:35 +0000 (0:00:00.006)       0:03:28.226 *******
2026-04-08 03:11:35.969725 | instance | ok: [instance]
2026-04-08 03:11:35.969825 | instance |
2026-04-08 03:11:35.969907 | instance | TASK [vexxhost.containers.forget_package : Forget package] *********************
2026-04-08 03:11:35.970031 | instance | Wednesday 08 April 2026  03:11:35 +0000 (0:00:00.472)       0:03:28.699 *******
2026-04-08 03:11:36.284150 | instance | ok: [instance]
2026-04-08 03:11:36.284225 | instance |
2026-04-08 03:11:36.284378 | instance | TASK [vexxhost.containers.download_artifact : Starting download of file] *******
2026-04-08 03:11:36.284501 | instance | Wednesday 08 April 2026  03:11:36 +0000 (0:00:00.314)       0:03:29.014 *******
2026-04-08 03:11:36.341700 | instance | ok: [instance] => {
2026-04-08 03:11:36.341861 | instance |     "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz"
2026-04-08 03:11:36.342005 | instance | }
2026-04-08 03:11:36.342101 | instance |
2026-04-08 03:11:36.342257 | instance | TASK [vexxhost.containers.download_artifact : Download item] *******************
2026-04-08 03:11:36.342347 | instance | Wednesday 08 April 2026  03:11:36 +0000 (0:00:00.057)       0:03:29.071 *******
2026-04-08 03:11:36.789325 | instance | ok: [instance]
2026-04-08 03:11:36.789438 | instance |
2026-04-08 03:11:36.789584 | instance | TASK [vexxhost.containers.download_artifact : Extract archive] *****************
2026-04-08 03:11:36.789708 | instance | Wednesday 08 April 2026  03:11:36 +0000 (0:00:00.447)       0:03:29.519 *******
2026-04-08 03:11:39.983820 | instance | ok: [instance]
2026-04-08 03:11:39.983888 | instance |
2026-04-08 03:11:39.984157 | instance | TASK [vexxhost.containers.docker : Install AppArmor packages] ******************
2026-04-08 03:11:39.984204 | instance | Wednesday 08 April 2026  03:11:39 +0000 (0:00:03.194)       0:03:32.713 *******
2026-04-08 03:11:41.414001 | instance | ok: [instance]
2026-04-08 03:11:41.414065 | instance |
2026-04-08 03:11:41.414351 | instance | TASK [vexxhost.containers.docker : Ensure group "docker" exists] ***************
2026-04-08 03:11:41.414392 | instance | Wednesday 08 April 2026  03:11:41 +0000 (0:00:01.430)       0:03:34.143 *******
2026-04-08 03:11:41.739513 | instance | ok: [instance]
2026-04-08 03:11:41.739593 | instance |
2026-04-08 03:11:41.739852 | instance | TASK [vexxhost.containers.docker : Create systemd service file for docker] *****
2026-04-08 03:11:41.739890 | instance | Wednesday 08 April 2026  03:11:41 +0000 (0:00:00.325)       0:03:34.469 *******
2026-04-08 03:11:42.287477 | instance | ok: [instance]
2026-04-08 03:11:42.287962 | instance |
2026-04-08 03:11:42.287983 | instance | TASK [vexxhost.containers.docker : Create folders for configuration] ***********
2026-04-08 03:11:42.287991 | instance | Wednesday 08 April 2026  03:11:42 +0000 (0:00:00.547)       0:03:35.017 *******
2026-04-08 03:11:43.191887 | instance | ok: [instance] => (item={'path': '/etc/docker'})
2026-04-08 03:11:43.192400 | instance | ok: [instance] => (item={'path': '/var/lib/docker', 'mode': '0o710'})
2026-04-08 03:11:43.192466 | instance | ok: [instance] => (item={'path': '/run/docker', 'mode': '0o711'})
2026-04-08 03:11:43.192665 | instance |
2026-04-08 03:11:43.192683 | instance | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ******
2026-04-08 03:11:43.192808 | instance | Wednesday 08 April 2026  03:11:43 +0000 (0:00:00.904)       0:03:35.921 *******
2026-04-08 03:11:43.735879 | instance | ok: [instance]
2026-04-08 03:11:43.736265 | instance |
2026-04-08 03:11:43.736275 | instance | TASK [vexxhost.containers.docker : Create docker daemon config file] ***********
2026-04-08 03:11:43.736438 | instance | Wednesday 08 April 2026  03:11:43 +0000 (0:00:00.544)       0:03:36.465 *******
2026-04-08 03:11:44.277288 | instance | ok: [instance]
2026-04-08 03:11:44.277366 | instance |
2026-04-08 03:11:44.277658 | instance | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************
2026-04-08 03:11:44.277673 | instance | Wednesday 08 April 2026  03:11:44 +0000 (0:00:00.534)       0:03:37.000 *******
2026-04-08 03:11:44.277796 | instance |
2026-04-08 03:11:44.277919 | instance | TASK [vexxhost.containers.docker : Enable and start service] *******************
2026-04-08 03:11:44.278060 | instance | Wednesday 08 April 2026  03:11:44 +0000 (0:00:00.006)       0:03:37.007 *******
2026-04-08 03:11:44.751324 | instance | ok: [instance]
2026-04-08 03:11:44.751519 | instance |
2026-04-08 03:11:44.751802 | instance | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ******
2026-04-08 03:11:44.752080 | instance | Wednesday 08 April 2026  03:11:44 +0000 (0:00:00.473)       0:03:37.480 *******
2026-04-08 03:11:44.807825 | instance | ok: [instance] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/ubuntu-22.04.yml)
2026-04-08 03:11:44.808065 | instance |
2026-04-08 03:11:44.808384 | instance | TASK [vexxhost.ceph.cephadm : Install packages] ********************************
2026-04-08 03:11:44.808718 | instance | Wednesday 08 April 2026  03:11:44 +0000 (0:00:00.056)       0:03:37.537 *******
2026-04-08 03:11:46.260906 | instance | ok: [instance]
2026-04-08 03:11:46.261180 | instance |
2026-04-08 03:11:46.261564 | instance | TASK [vexxhost.ceph.cephadm : Ensure services are started] *********************
2026-04-08 03:11:46.261891 | instance | Wednesday 08 April 2026  03:11:46 +0000 (0:00:01.453)       0:03:38.990 *******
2026-04-08 03:11:47.169225 | instance | ok: [instance] => (item=chronyd)
2026-04-08 03:11:47.169412 | instance | ok: [instance] => (item=sshd)
2026-04-08 03:11:47.169632 | instance |
2026-04-08 03:11:47.169922 | instance | TASK [vexxhost.ceph.cephadm : Download "cephadm"] ******************************
2026-04-08 03:11:47.170154 | instance | Wednesday 08 April 2026  03:11:47 +0000 (0:00:00.908)       0:03:39.898 *******
2026-04-08 03:11:47.612071 | instance | ok: [instance]
2026-04-08 03:11:47.612369 | instance |
2026-04-08 03:11:47.612679 | instance | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ********************
2026-04-08 03:11:47.612968 | instance | Wednesday 08 April 2026  03:11:47 +0000 (0:00:00.442)       0:03:40.341 *******
2026-04-08 03:11:47.948923 | instance | ok: [instance]
2026-04-08 03:11:47.949204 | instance |
2026-04-08 03:11:47.949520 | instance | TASK [vexxhost.ceph.cephadm : Ensure "cephadm" user is present] ****************
2026-04-08 03:11:47.949799 | instance | Wednesday 08 April 2026  03:11:47 +0000 (0:00:00.336)       0:03:40.677 *******
2026-04-08 03:11:48.333572 | instance | ok: [instance]
2026-04-08 03:11:48.333735 | instance |
2026-04-08 03:11:48.334017 | instance | TASK [vexxhost.ceph.cephadm : Allow "cephadm" user to have passwordless sudo] ***
2026-04-08 03:11:48.334289 | instance | Wednesday 08 April 2026  03:11:48 +0000 (0:00:00.385)       0:03:41.063 *******
2026-04-08 03:11:48.676517 | instance | ok: [instance]
2026-04-08 03:11:48.676627 | instance |
2026-04-08 03:11:48.676641 | instance | TASK [vexxhost.ceph.osd : Get monitor status] **********************************
2026-04-08 03:11:48.676660 | instance | Wednesday 08 April 2026  03:11:48 +0000 (0:00:00.342)       0:03:41.405 *******
2026-04-08 03:11:49.046180 | instance | ok: [instance] => (item=instance)
2026-04-08 03:11:49.046302 | instance |
2026-04-08 03:11:49.046581 | instance | TASK [vexxhost.ceph.osd : Select admin host] ***********************************
2026-04-08 03:11:49.046626 | instance | Wednesday 08 April 2026  03:11:49 +0000 (0:00:00.370)       0:03:41.776 *******
2026-04-08 03:11:49.102906 | instance | ok: [instance]
2026-04-08 03:11:49.103025 | instance |
2026-04-08 03:11:49.103398 | instance | TASK [vexxhost.ceph.osd : Get `cephadm ls` status] *****************************
2026-04-08 03:11:49.103433 | instance | Wednesday 08 April 2026  03:11:49 +0000 (0:00:00.056)       0:03:41.832 *******
2026-04-08 03:11:54.530438 | instance | ok: [instance]
2026-04-08 03:11:54.530516 | instance |
2026-04-08 03:11:54.530763 | instance | TASK [vexxhost.ceph.osd : Parse the `cephadm ls` output] ***********************
2026-04-08 03:11:54.530809 | instance | Wednesday 08 April 2026  03:11:54 +0000 (0:00:05.427)       0:03:47.260 *******
2026-04-08 03:11:54.584158 | instance | ok: [instance]
2026-04-08 03:11:54.584365 | instance |
2026-04-08 03:11:54.584423 | instance | TASK [Install Ceph host] *******************************************************
2026-04-08 03:11:54.584532 | instance | Wednesday 08 April 2026  03:11:54 +0000 (0:00:00.053)       0:03:47.314 *******
2026-04-08 03:11:54.654181 | instance | included: vexxhost.ceph.cephadm_host for instance
2026-04-08 03:11:54.654275 | instance |
2026-04-08 03:11:54.654408 | instance | TASK [vexxhost.ceph.cephadm_host : Get public SSH key for "cephadm" user] ******
2026-04-08 03:11:54.654540 | instance | Wednesday 08 April 2026  03:11:54 +0000 (0:00:00.069)       0:03:47.383 *******
2026-04-08 03:11:54.706024 | instance | skipping: [instance]
2026-04-08 03:11:54.706242 | instance |
2026-04-08 03:11:54.706523 | instance | TASK [vexxhost.ceph.cephadm_host : Set fact with public SSH key for "cephadm" user] ***
2026-04-08 03:11:54.706783 | instance | Wednesday 08 April 2026  03:11:54 +0000 (0:00:00.051)       0:03:47.435 *******
2026-04-08 03:11:54.752711 | instance | skipping: [instance] => (item=instance)
2026-04-08 03:11:54.752946 | instance | skipping: [instance]
2026-04-08 03:11:54.753240 | instance |
2026-04-08 03:11:54.753518 | instance | TASK [vexxhost.ceph.cephadm_host : Set authorized key for "cephadm"] ***********
2026-04-08 03:11:54.753842 | instance | Wednesday 08 April 2026  03:11:54 +0000 (0:00:00.047)       0:03:47.482 *******
2026-04-08 03:11:55.112396 | instance | ok: [instance]
2026-04-08 03:11:55.112631 | instance |
2026-04-08 03:11:55.112896 | instance | TASK [vexxhost.ceph.cephadm_host : Add new host to Ceph] ***********************
2026-04-08 03:11:55.113167 | instance | Wednesday 08 April 2026  03:11:55 +0000 (0:00:00.359)       0:03:47.842 *******
2026-04-08 03:11:57.243917 | instance | ok: [instance]
2026-04-08 03:11:57.244111 | instance |
2026-04-08 03:11:57.244399 | instance | TASK [vexxhost.ceph.osd : Adopt OSDs to cluster] *******************************
2026-04-08 03:11:57.244649 | instance | Wednesday 08 April 2026  03:11:57 +0000 (0:00:02.131)       0:03:49.973 *******
2026-04-08 03:11:57.274193 | instance | skipping: [instance]
2026-04-08 03:11:57.274447 | instance |
2026-04-08 03:11:57.274704 | instance | TASK [vexxhost.ceph.osd : Wait until OSD added to cephadm] *********************
2026-04-08 03:11:57.274972 | instance | Wednesday 08 April 2026  03:11:57 +0000 (0:00:00.030)       0:03:50.004 *******
2026-04-08 03:11:57.308824 | instance | skipping: [instance]
2026-04-08 03:11:57.309074 | instance |
2026-04-08 03:11:57.309345 | instance | TASK [vexxhost.ceph.osd : Ensure all OSDs are non-legacy] **********************
2026-04-08 03:11:57.309615 | instance | Wednesday 08 April 2026  03:11:57 +0000 (0:00:00.034)       0:03:50.038 *******
2026-04-08 03:12:02.804395 | instance | ok: [instance]
2026-04-08 03:12:02.805178 | instance |
2026-04-08 03:12:02.805242 | instance | TASK [vexxhost.ceph.osd : Get `ceph-volume lvm list` status] *******************
2026-04-08 03:12:02.805258 | instance | Wednesday 08 April 2026  03:12:02 +0000 (0:00:05.494)       0:03:55.533 *******
2026-04-08 03:12:13.262917 | instance | ok: [instance]
2026-04-08 03:12:13.263052 | instance |
2026-04-08 03:12:13.263410 | instance | TASK [vexxhost.ceph.osd : Install OSDs] ****************************************
2026-04-08 03:12:13.263453 | instance | Wednesday 08 April 2026  03:12:13 +0000 (0:00:10.458)       0:04:05.992 *******
2026-04-08 03:12:21.208050 | instance | failed: [instance] (item=/dev/vdb) => {"ansible_loop_var": "item", "changed": false, "cmd": ["cephadm", "shell", "--fsid", "4837cbf8-4f90-4300-b3f6-726c9b9f89b4", "--config", "/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config", "--", "ceph", "orch", "daemon", "add", "osd", "instance:/dev/vdb"], "delta": "0:00:07.543346", "end": "2026-04-08 03:12:21.159991", "item": "/dev/vdb", "msg": "non-zero return code", "rc": 22, "start": "2026-04-08 03:12:13.616645", "stderr": "Error EINVAL: Traceback (most recent call last):\n  File \"/usr/share/ceph/mgr/mgr_module.py\", line 1834, in _handle_command\n    return self.handle_command(inbuf, cmd)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 183, in handle_command\n    return dispatch[cmd['prefix']].call(self, cmd, inbuf)\n  File \"/usr/share/ceph/mgr/mgr_module.py\", line 475, in call\n    return self.func(mgr, **kwargs)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 119, in <lambda>\n    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 108, in wrapper\n    return func(*args, **kwargs)\n  File \"/usr/share/ceph/mgr/orchestrator/module.py\", line 1306, in _daemon_add_osd\n    raise_if_exception(completion)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 240, in raise_if_exception\n    raise e\nRuntimeError: cephadm exited with an error code: 1, stderr:Inferring config /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config\nNon-zero exit code 1 from /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpl_l6kdp1:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmp2os61re8:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdb --yes --no-systemd\n/usr/bin/docker: stderr  stderr: lsblk: /dev/vdb: not a block device\n/usr/bin/docker: stderr Traceback (most recent call last):\n/usr/bin/docker: stderr   File \"/usr/sbin/ceph-volume\", line 33, in <module>\n/usr/bin/docker: stderr     sys.exit(load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')())\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 54, in __init__\n/usr/bin/docker: stderr     self.main(self.argv)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/decorators.py\", line 59, in newfunc\n/usr/bin/docker: stderr     return f(*a, **kw)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 166, in main\n/usr/bin/docker: stderr     terminal.dispatch(self.mapper, subcommand_args)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 194, in dispatch\n/usr/bin/docker: stderr     instance.main()\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/main.py\", line 46, in main\n/usr/bin/docker: stderr     terminal.dispatch(self.mapper, self.argv)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 192, in dispatch\n/usr/bin/docker: stderr     instance = mapper.get(arg)(argv[count:])\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/batch.py\", line 325, in __init__\n/usr/bin/docker: stderr     self.args = parser.parse_args(argv)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1825, in parse_args\n/usr/bin/docker: stderr     args, argv = self.parse_known_args(args, namespace)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1858, in parse_known_args\n/usr/bin/docker: stderr     namespace, args = self._parse_known_args(args, namespace)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2049, in _parse_known_args\n/usr/bin/docker: stderr     positionals_end_index = consume_positionals(start_index)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2026, in consume_positionals\n/usr/bin/docker: stderr     take_action(action, args)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1919, in take_action\n/usr/bin/docker: stderr     argument_values = self._get_values(action, argument_strings)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in _get_values\n/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in <listcomp>\n/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2483, in _get_value\n/usr/bin/docker: stderr     result = type_func(arg_string)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 124, in __call__\n/usr/bin/docker: stderr     super().get_device(dev_path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 32, in get_device\n/usr/bin/docker: stderr     self._device = Device(dev_path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 140, in __init__\n/usr/bin/docker: stderr     self._parse()\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 236, in _parse\n/usr/bin/docker: stderr     dev = disk.lsblk(self.path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 244, in lsblk\n/usr/bin/docker: stderr     result = lsblk_all(device=device,\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 338, in lsblk_all\n/usr/bin/docker: stderr     raise RuntimeError(f\"Error: {err}\")\n/usr/bin/docker: stderr RuntimeError: Error: ['lsblk: /dev/vdb: not a block device']\nTraceback (most recent call last):\n  File \"/usr/lib/python3.10/runpy.py\", line 196, in _run_module_as_main\n    return _run_code(code, main_globals, None,\n  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code\n    exec(code, run_globals)\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 11009, in <module>\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 10997, in main\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2593, in _infer_config\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2509, in _infer_fsid\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2621, in _infer_image\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2496, in _validate_fsid\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 7226, in command_ceph_volume\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2284, in call_throws\nRuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpl_l6kdp1:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmp2os61re8:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdb --yes --no-systemd", "stderr_lines": ["Error EINVAL: Traceback (most recent call last):", "  File \"/usr/share/ceph/mgr/mgr_module.py\", line 1834, in _handle_command", "    return self.handle_command(inbuf, cmd)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 183, in handle_command", "    return dispatch[cmd['prefix']].call(self, cmd, inbuf)", "  File \"/usr/share/ceph/mgr/mgr_module.py\", line 475, in call", "    return self.func(mgr, **kwargs)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 119, in <lambda>", "    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 108, in wrapper", "    return func(*args, **kwargs)", "  File \"/usr/share/ceph/mgr/orchestrator/module.py\", line 1306, in _daemon_add_osd", "    raise_if_exception(completion)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 240, in raise_if_exception", "    raise e", "RuntimeError: cephadm exited with an error code: 1, stderr:Inferring config /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config", "Non-zero exit code 1 from /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpl_l6kdp1:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmp2os61re8:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdb --yes --no-systemd", "/usr/bin/docker: stderr  stderr: lsblk: /dev/vdb: not a block device", "/usr/bin/docker: stderr Traceback (most recent call last):", "/usr/bin/docker: stderr   File \"/usr/sbin/ceph-volume\", line 33, in <module>", "/usr/bin/docker: stderr     sys.exit(load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')())", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 54, in __init__", "/usr/bin/docker: stderr     self.main(self.argv)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/decorators.py\", line 59, in newfunc", "/usr/bin/docker: stderr     return f(*a, **kw)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 166, in main", "/usr/bin/docker: stderr     terminal.dispatch(self.mapper, subcommand_args)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 194, in dispatch", "/usr/bin/docker: stderr     instance.main()", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/main.py\", line 46, in main", "/usr/bin/docker: stderr     terminal.dispatch(self.mapper, self.argv)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 192, in dispatch", "/usr/bin/docker: stderr     instance = mapper.get(arg)(argv[count:])", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/batch.py\", line 325, in __init__", "/usr/bin/docker: stderr     self.args = parser.parse_args(argv)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1825, in parse_args", "/usr/bin/docker: stderr     args, argv = self.parse_known_args(args, namespace)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1858, in parse_known_args", "/usr/bin/docker: stderr     namespace, args = self._parse_known_args(args, namespace)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2049, in _parse_known_args", "/usr/bin/docker: stderr     positionals_end_index = consume_positionals(start_index)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2026, in consume_positionals", "/usr/bin/docker: stderr     take_action(action, args)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1919, in take_action", "/usr/bin/docker: stderr     argument_values = self._get_values(action, argument_strings)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in _get_values", "/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in <listcomp>", "/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2483, in _get_value", "/usr/bin/docker: stderr     result = type_func(arg_string)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 124, in __call__", "/usr/bin/docker: stderr     super().get_device(dev_path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 32, in get_device", "/usr/bin/docker: stderr     self._device = Device(dev_path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 140, in __init__", "/usr/bin/docker: stderr     self._parse()", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 236, in _parse", "/usr/bin/docker: stderr     dev = disk.lsblk(self.path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 244, in lsblk", "/usr/bin/docker: stderr     result = lsblk_all(device=device,", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 338, in lsblk_all", "/usr/bin/docker: stderr     raise RuntimeError(f\"Error: {err}\")", "/usr/bin/docker: stderr RuntimeError: Error: ['lsblk: /dev/vdb: not a block device']", "Traceback (most recent call last):", "  File \"/usr/lib/python3.10/runpy.py\", line 196, in _run_module_as_main", "    return _run_code(code, main_globals, None,", "  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code", "    exec(code, run_globals)", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 11009, in <module>", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 10997, in main", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2593, in _infer_config", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2509, in _infer_fsid", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2621, in _infer_image", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2496, in _validate_fsid", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 7226, in command_ceph_volume", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2284, in call_throws", "RuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpl_l6kdp1:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmp2os61re8:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdb --yes --no-systemd"], "stdout": "", "stdout_lines": []}
2026-04-08 03:12:29.070732 | instance | failed: [instance] (item=/dev/vdc) => {"ansible_loop_var": "item", "changed": false, "cmd": ["cephadm", "shell", "--fsid", "4837cbf8-4f90-4300-b3f6-726c9b9f89b4", "--config", "/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config", "--", "ceph", "orch", "daemon", "add", "osd", "instance:/dev/vdc"], "delta": "0:00:07.570282", "end": "2026-04-08 03:12:29.023270", "item": "/dev/vdc", "msg": "non-zero return code", "rc": 22, "start": "2026-04-08 03:12:21.452988", "stderr": "Error EINVAL: Traceback (most recent call last):\n  File \"/usr/share/ceph/mgr/mgr_module.py\", line 1834, in _handle_command\n    return self.handle_command(inbuf, cmd)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 183, in handle_command\n    return dispatch[cmd['prefix']].call(self, cmd, inbuf)\n  File \"/usr/share/ceph/mgr/mgr_module.py\", line 475, in call\n    return self.func(mgr, **kwargs)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 119, in <lambda>\n    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 108, in wrapper\n    return func(*args, **kwargs)\n  File \"/usr/share/ceph/mgr/orchestrator/module.py\", line 1306, in _daemon_add_osd\n    raise_if_exception(completion)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 240, in raise_if_exception\n    raise e\nRuntimeError: cephadm exited with an error code: 1, stderr:Inferring config /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config\nNon-zero exit code 1 from /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpomfz0pb4:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmprk4rqtt_:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdc --yes --no-systemd\n/usr/bin/docker: stderr  stderr: lsblk: /dev/vdc: not a block device\n/usr/bin/docker: stderr Traceback (most recent call last):\n/usr/bin/docker: stderr   File \"/usr/sbin/ceph-volume\", line 33, in <module>\n/usr/bin/docker: stderr     sys.exit(load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')())\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 54, in __init__\n/usr/bin/docker: stderr     self.main(self.argv)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/decorators.py\", line 59, in newfunc\n/usr/bin/docker: stderr     return f(*a, **kw)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 166, in main\n/usr/bin/docker: stderr     terminal.dispatch(self.mapper, subcommand_args)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 194, in dispatch\n/usr/bin/docker: stderr     instance.main()\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/main.py\", line 46, in main\n/usr/bin/docker: stderr     terminal.dispatch(self.mapper, self.argv)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 192, in dispatch\n/usr/bin/docker: stderr     instance = mapper.get(arg)(argv[count:])\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/batch.py\", line 325, in __init__\n/usr/bin/docker: stderr     self.args = parser.parse_args(argv)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1825, in parse_args\n/usr/bin/docker: stderr     args, argv = self.parse_known_args(args, namespace)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1858, in parse_known_args\n/usr/bin/docker: stderr     namespace, args = self._parse_known_args(args, namespace)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2049, in _parse_known_args\n/usr/bin/docker: stderr     positionals_end_index = consume_positionals(start_index)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2026, in consume_positionals\n/usr/bin/docker: stderr     take_action(action, args)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1919, in take_action\n/usr/bin/docker: stderr     argument_values = self._get_values(action, argument_strings)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in _get_values\n/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in <listcomp>\n/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2483, in _get_value\n/usr/bin/docker: stderr     result = type_func(arg_string)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 124, in __call__\n/usr/bin/docker: stderr     super().get_device(dev_path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 32, in get_device\n/usr/bin/docker: stderr     self._device = Device(dev_path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 140, in __init__\n/usr/bin/docker: stderr     self._parse()\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 236, in _parse\n/usr/bin/docker: stderr     dev = disk.lsblk(self.path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 244, in lsblk\n/usr/bin/docker: stderr     result = lsblk_all(device=device,\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 338, in lsblk_all\n/usr/bin/docker: stderr     raise RuntimeError(f\"Error: {err}\")\n/usr/bin/docker: stderr RuntimeError: Error: ['lsblk: /dev/vdc: not a block device']\nTraceback (most recent call last):\n  File \"/usr/lib/python3.10/runpy.py\", line 196, in _run_module_as_main\n    return _run_code(code, main_globals, None,\n  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code\n    exec(code, run_globals)\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 11009, in <module>\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 10997, in main\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2593, in _infer_config\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2509, in _infer_fsid\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2621, in _infer_image\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2496, in _validate_fsid\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 7226, in command_ceph_volume\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2284, in call_throws\nRuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpomfz0pb4:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmprk4rqtt_:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdc --yes --no-systemd", "stderr_lines": ["Error EINVAL: Traceback (most recent call last):", "  File \"/usr/share/ceph/mgr/mgr_module.py\", line 1834, in _handle_command", "    return self.handle_command(inbuf, cmd)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 183, in handle_command", "    return dispatch[cmd['prefix']].call(self, cmd, inbuf)", "  File \"/usr/share/ceph/mgr/mgr_module.py\", line 475, in call", "    return self.func(mgr, **kwargs)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 119, in <lambda>", "    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 108, in wrapper", "    return func(*args, **kwargs)", "  File \"/usr/share/ceph/mgr/orchestrator/module.py\", line 1306, in _daemon_add_osd", "    raise_if_exception(completion)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 240, in raise_if_exception", "    raise e", "RuntimeError: cephadm exited with an error code: 1, stderr:Inferring config /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config", "Non-zero exit code 1 from /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpomfz0pb4:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmprk4rqtt_:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdc --yes --no-systemd", "/usr/bin/docker: stderr  stderr: lsblk: /dev/vdc: not a block device", "/usr/bin/docker: stderr Traceback (most recent call last):", "/usr/bin/docker: stderr   File \"/usr/sbin/ceph-volume\", line 33, in <module>", "/usr/bin/docker: stderr     sys.exit(load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')())", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 54, in __init__", "/usr/bin/docker: stderr     self.main(self.argv)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/decorators.py\", line 59, in newfunc", "/usr/bin/docker: stderr     return f(*a, **kw)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 166, in main", "/usr/bin/docker: stderr     terminal.dispatch(self.mapper, subcommand_args)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 194, in dispatch", "/usr/bin/docker: stderr     instance.main()", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/main.py\", line 46, in main", "/usr/bin/docker: stderr     terminal.dispatch(self.mapper, self.argv)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 192, in dispatch", "/usr/bin/docker: stderr     instance = mapper.get(arg)(argv[count:])", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/batch.py\", line 325, in __init__", "/usr/bin/docker: stderr     self.args = parser.parse_args(argv)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1825, in parse_args", "/usr/bin/docker: stderr     args, argv = self.parse_known_args(args, namespace)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1858, in parse_known_args", "/usr/bin/docker: stderr     namespace, args = self._parse_known_args(args, namespace)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2049, in _parse_known_args", "/usr/bin/docker: stderr     positionals_end_index = consume_positionals(start_index)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2026, in consume_positionals", "/usr/bin/docker: stderr     take_action(action, args)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1919, in take_action", "/usr/bin/docker: stderr     argument_values = self._get_values(action, argument_strings)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in _get_values", "/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in <listcomp>", "/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2483, in _get_value", "/usr/bin/docker: stderr     result = type_func(arg_string)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 124, in __call__", "/usr/bin/docker: stderr     super().get_device(dev_path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 32, in get_device", "/usr/bin/docker: stderr     self._device = Device(dev_path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 140, in __init__", "/usr/bin/docker: stderr     self._parse()", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 236, in _parse", "/usr/bin/docker: stderr     dev = disk.lsblk(self.path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 244, in lsblk", "/usr/bin/docker: stderr     result = lsblk_all(device=device,", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 338, in lsblk_all", "/usr/bin/docker: stderr     raise RuntimeError(f\"Error: {err}\")", "/usr/bin/docker: stderr RuntimeError: Error: ['lsblk: /dev/vdc: not a block device']", "Traceback (most recent call last):", "  File \"/usr/lib/python3.10/runpy.py\", line 196, in _run_module_as_main", "    return _run_code(code, main_globals, None,", "  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code", "    exec(code, run_globals)", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 11009, in <module>", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 10997, in main", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2593, in _infer_config", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2509, in _infer_fsid", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2621, in _infer_image", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2496, in _validate_fsid", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 7226, in command_ceph_volume", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2284, in call_throws", "RuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpomfz0pb4:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmprk4rqtt_:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdc --yes --no-systemd"], "stdout": "", "stdout_lines": []}
2026-04-08 03:12:36.978672 | instance | failed: [instance] (item=/dev/vdd) => {"ansible_loop_var": "item", "changed": false, "cmd": ["cephadm", "shell", "--fsid", "4837cbf8-4f90-4300-b3f6-726c9b9f89b4", "--config", "/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config", "--", "ceph", "orch", "daemon", "add", "osd", "instance:/dev/vdd"], "delta": "0:00:07.607305", "end": "2026-04-08 03:12:36.920323", "item": "/dev/vdd", "msg": "non-zero return code", "rc": 22, "start": "2026-04-08 03:12:29.313018", "stderr": "Error EINVAL: Traceback (most recent call last):\n  File \"/usr/share/ceph/mgr/mgr_module.py\", line 1834, in _handle_command\n    return self.handle_command(inbuf, cmd)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 183, in handle_command\n    return dispatch[cmd['prefix']].call(self, cmd, inbuf)\n  File \"/usr/share/ceph/mgr/mgr_module.py\", line 475, in call\n    return self.func(mgr, **kwargs)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 119, in <lambda>\n    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 108, in wrapper\n    return func(*args, **kwargs)\n  File \"/usr/share/ceph/mgr/orchestrator/module.py\", line 1306, in _daemon_add_osd\n    raise_if_exception(completion)\n  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 240, in raise_if_exception\n    raise e\nRuntimeError: cephadm exited with an error code: 1, stderr:Inferring config /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config\nNon-zero exit code 1 from /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpv5l0d6z9:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmpy41nqju2:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdd --yes --no-systemd\n/usr/bin/docker: stderr  stderr: lsblk: /dev/vdd: not a block device\n/usr/bin/docker: stderr Traceback (most recent call last):\n/usr/bin/docker: stderr   File \"/usr/sbin/ceph-volume\", line 33, in <module>\n/usr/bin/docker: stderr     sys.exit(load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')())\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 54, in __init__\n/usr/bin/docker: stderr     self.main(self.argv)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/decorators.py\", line 59, in newfunc\n/usr/bin/docker: stderr     return f(*a, **kw)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 166, in main\n/usr/bin/docker: stderr     terminal.dispatch(self.mapper, subcommand_args)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 194, in dispatch\n/usr/bin/docker: stderr     instance.main()\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/main.py\", line 46, in main\n/usr/bin/docker: stderr     terminal.dispatch(self.mapper, self.argv)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 192, in dispatch\n/usr/bin/docker: stderr     instance = mapper.get(arg)(argv[count:])\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/batch.py\", line 325, in __init__\n/usr/bin/docker: stderr     self.args = parser.parse_args(argv)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1825, in parse_args\n/usr/bin/docker: stderr     args, argv = self.parse_known_args(args, namespace)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1858, in parse_known_args\n/usr/bin/docker: stderr     namespace, args = self._parse_known_args(args, namespace)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2049, in _parse_known_args\n/usr/bin/docker: stderr     positionals_end_index = consume_positionals(start_index)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2026, in consume_positionals\n/usr/bin/docker: stderr     take_action(action, args)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1919, in take_action\n/usr/bin/docker: stderr     argument_values = self._get_values(action, argument_strings)\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in _get_values\n/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in <listcomp>\n/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]\n/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2483, in _get_value\n/usr/bin/docker: stderr     result = type_func(arg_string)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 124, in __call__\n/usr/bin/docker: stderr     super().get_device(dev_path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 32, in get_device\n/usr/bin/docker: stderr     self._device = Device(dev_path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 140, in __init__\n/usr/bin/docker: stderr     self._parse()\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 236, in _parse\n/usr/bin/docker: stderr     dev = disk.lsblk(self.path)\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 244, in lsblk\n/usr/bin/docker: stderr     result = lsblk_all(device=device,\n/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 338, in lsblk_all\n/usr/bin/docker: stderr     raise RuntimeError(f\"Error: {err}\")\n/usr/bin/docker: stderr RuntimeError: Error: ['lsblk: /dev/vdd: not a block device']\nTraceback (most recent call last):\n  File \"/usr/lib/python3.10/runpy.py\", line 196, in _run_module_as_main\n    return _run_code(code, main_globals, None,\n  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code\n    exec(code, run_globals)\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 11009, in <module>\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 10997, in main\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2593, in _infer_config\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2509, in _infer_fsid\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2621, in _infer_image\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2496, in _validate_fsid\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 7226, in command_ceph_volume\n  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2284, in call_throws\nRuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpv5l0d6z9:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmpy41nqju2:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdd --yes --no-systemd", "stderr_lines": ["Error EINVAL: Traceback (most recent call last):", "  File \"/usr/share/ceph/mgr/mgr_module.py\", line 1834, in _handle_command", "    return self.handle_command(inbuf, cmd)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 183, in handle_command", "    return dispatch[cmd['prefix']].call(self, cmd, inbuf)", "  File \"/usr/share/ceph/mgr/mgr_module.py\", line 475, in call", "    return self.func(mgr, **kwargs)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 119, in <lambda>", "    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 108, in wrapper", "    return func(*args, **kwargs)", "  File \"/usr/share/ceph/mgr/orchestrator/module.py\", line 1306, in _daemon_add_osd", "    raise_if_exception(completion)", "  File \"/usr/share/ceph/mgr/orchestrator/_interface.py\", line 240, in raise_if_exception", "    raise e", "RuntimeError: cephadm exited with an error code: 1, stderr:Inferring config /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/mon.instance/config", "Non-zero exit code 1 from /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpv5l0d6z9:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmpy41nqju2:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdd --yes --no-systemd", "/usr/bin/docker: stderr  stderr: lsblk: /dev/vdd: not a block device", "/usr/bin/docker: stderr Traceback (most recent call last):", "/usr/bin/docker: stderr   File \"/usr/sbin/ceph-volume\", line 33, in <module>", "/usr/bin/docker: stderr     sys.exit(load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')())", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 54, in __init__", "/usr/bin/docker: stderr     self.main(self.argv)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/decorators.py\", line 59, in newfunc", "/usr/bin/docker: stderr     return f(*a, **kw)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/main.py\", line 166, in main", "/usr/bin/docker: stderr     terminal.dispatch(self.mapper, subcommand_args)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 194, in dispatch", "/usr/bin/docker: stderr     instance.main()", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/main.py\", line 46, in main", "/usr/bin/docker: stderr     terminal.dispatch(self.mapper, self.argv)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/terminal.py\", line 192, in dispatch", "/usr/bin/docker: stderr     instance = mapper.get(arg)(argv[count:])", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/devices/lvm/batch.py\", line 325, in __init__", "/usr/bin/docker: stderr     self.args = parser.parse_args(argv)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1825, in parse_args", "/usr/bin/docker: stderr     args, argv = self.parse_known_args(args, namespace)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1858, in parse_known_args", "/usr/bin/docker: stderr     namespace, args = self._parse_known_args(args, namespace)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2049, in _parse_known_args", "/usr/bin/docker: stderr     positionals_end_index = consume_positionals(start_index)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2026, in consume_positionals", "/usr/bin/docker: stderr     take_action(action, args)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 1919, in take_action", "/usr/bin/docker: stderr     argument_values = self._get_values(action, argument_strings)", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in _get_values", "/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2468, in <listcomp>", "/usr/bin/docker: stderr     value = [self._get_value(action, v) for v in arg_strings]", "/usr/bin/docker: stderr   File \"/usr/lib64/python3.9/argparse.py\", line 2483, in _get_value", "/usr/bin/docker: stderr     result = type_func(arg_string)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 124, in __call__", "/usr/bin/docker: stderr     super().get_device(dev_path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/arg_validators.py\", line 32, in get_device", "/usr/bin/docker: stderr     self._device = Device(dev_path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 140, in __init__", "/usr/bin/docker: stderr     self._parse()", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/device.py\", line 236, in _parse", "/usr/bin/docker: stderr     dev = disk.lsblk(self.path)", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 244, in lsblk", "/usr/bin/docker: stderr     result = lsblk_all(device=device,", "/usr/bin/docker: stderr   File \"/usr/lib/python3.9/site-packages/ceph_volume/util/disk.py\", line 338, in lsblk_all", "/usr/bin/docker: stderr     raise RuntimeError(f\"Error: {err}\")", "/usr/bin/docker: stderr RuntimeError: Error: ['lsblk: /dev/vdd: not a block device']", "Traceback (most recent call last):", "  File \"/usr/lib/python3.10/runpy.py\", line 196, in _run_module_as_main", "    return _run_code(code, main_globals, None,", "  File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code", "    exec(code, run_globals)", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 11009, in <module>", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 10997, in main", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2593, in _infer_config", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2509, in _infer_fsid", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2621, in _infer_image", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2496, in _validate_fsid", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 7226, in command_ceph_volume", "  File \"/var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/cephadm.31206ab20142c8051b6384b731ef7ef7af2407447fac35b7291e90720452ed8d/__main__.py\", line 2284, in call_throws", "RuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 -e NODE_NAME=instance -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/run/ceph:z -v /var/log/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4:/var/log/ceph:z -v /var/lib/ceph/4837cbf8-4f90-4300-b3f6-726c9b9f89b4/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmpv5l0d6z9:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmpy41nqju2:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:1b9158ce28975f95def6a0ad459fa19f1336506074267a4b47c1bd914a00fec0 lvm batch --no-auto /dev/vdd --yes --no-systemd"], "stdout": "", "stdout_lines": []}
2026-04-08 03:12:36.983525 | instance |
2026-04-08 03:12:36.983702 | instance | PLAY RECAP *********************************************************************
2026-04-08 03:12:36.984456 | instance | instance                   : ok=105  changed=26   unreachable=0    failed=1    skipped=26   rescued=0    ignored=0
2026-04-08 03:12:36.984570 | instance |
2026-04-08 03:12:36.984584 | instance | Wednesday 08 April 2026  03:12:36 +0000 (0:00:23.721)       0:04:29.713 *******
2026-04-08 03:12:36.984606 | instance | ===============================================================================
2026-04-08 03:12:36.984809 | instance | vexxhost.ceph.mon : Run Bootstrap coomand ----------------------------- 126.19s
2026-04-08 03:12:36.985195 | instance | vexxhost.ceph.osd : Install OSDs --------------------------------------- 23.72s
2026-04-08 03:12:36.985238 | instance | vexxhost.ceph.osd : Get `ceph-volume lvm list` status ------------------ 10.46s
2026-04-08 03:12:36.985247 | instance | vexxhost.ceph.mon : Validate monitor exist ----------------------------- 10.37s
2026-04-08 03:12:36.987746 | instance | vexxhost.ceph.osd : Ensure all OSDs are non-legacy ---------------------- 5.50s
2026-04-08 03:12:36.987792 | instance | vexxhost.ceph.osd : Get `cephadm ls` status ----------------------------- 5.43s
2026-04-08 03:12:36.987800 | instance | vexxhost.ceph.cephadm : Install packages -------------------------------- 5.41s
2026-04-08 03:12:36.987808 | instance | vexxhost.containers.containerd : Install AppArmor packages -------------- 5.38s
2026-04-08 03:12:36.987814 | instance | vexxhost.containers.download_artifact : Extract archive ----------------- 4.43s
2026-04-08 03:12:36.987820 | instance | vexxhost.containers.download_artifact : Extract archive ----------------- 3.19s
2026-04-08 03:12:36.987825 | instance | vexxhost.containers.download_artifact : Extract archive ----------------- 3.01s
2026-04-08 03:12:36.987831 | instance | vexxhost.ceph.cephadm_host : Add new host to Ceph ----------------------- 2.13s
2026-04-08 03:12:36.987837 | instance | vexxhost.ceph.cephadm_host : Add new host to Ceph ----------------------- 2.11s
2026-04-08 03:12:36.987842 | instance | vexxhost.containers.download_artifact : Extract archive ----------------- 2.04s
2026-04-08 03:12:36.987848 | instance | vexxhost.ceph.cephadm_host : Add new host to Ceph ----------------------- 2.03s
2026-04-08 03:12:36.987854 | instance | vexxhost.containers.containerd : Reload systemd ------------------------- 1.94s
2026-04-08 03:12:36.987859 | instance | vexxhost.ceph.mgr : Enable the Ceph Manager prometheus module ----------- 1.92s
2026-04-08 03:12:36.987865 | instance | vexxhost.ceph.mon : Get `cephadm ls` status ----------------------------- 1.81s
2026-04-08 03:12:36.987871 | instance | vexxhost.ceph.mon : Configure "mon" label for monitors ------------------ 1.80s
2026-04-08 03:12:36.987877 | instance | vexxhost.ceph.mgr : Configure "mgr" label for managers ------------------ 1.75s
2026-04-08 03:12:37.325226 | instance | CRITICAL Ansible return code was 2, command was: ansible-playbook --inventory /home/zuul/.ansible/tmp/molecule.v9Wo.aio/inventory --skip-tags molecule-notest,notest --inventory=/home/zuul/src/github.com/vexxhost/atmosphere/inventory.yaml /home/zuul/src/github.com/vexxhost/atmosphere/molecule/aio/converge.yml
2026-04-08 03:12:37.325389 | instance | ERROR    [aio > converge] Executed: Failed
2026-04-08 03:12:37.325483 | instance | ERROR    Ansible return code was 2, command was: ansible-playbook --inventory /home/zuul/.ansible/tmp/molecule.v9Wo.aio/inventory --skip-tags molecule-notest,notest --inventory=/home/zuul/src/github.com/vexxhost/atmosphere/inventory.yaml /home/zuul/src/github.com/vexxhost/atmosphere/molecule/aio/converge.yml
2026-04-08 03:12:37.666262 | instance | ERROR
2026-04-08 03:12:37.666555 | instance | {
2026-04-08 03:12:37.666619 | instance |   "delta": "0:06:16.182258",
2026-04-08 03:12:37.666670 | instance |   "end": "2026-04-08 03:12:37.402737",
2026-04-08 03:12:37.666712 | instance |   "msg": "non-zero return code",
2026-04-08 03:12:37.666752 | instance |   "rc": 2,
2026-04-08 03:12:37.666800 | instance |   "start": "2026-04-08 03:06:21.220479"
2026-04-08 03:12:37.666978 | instance | }
failure
2026-04-08 03:12:37.679555 | 
2026-04-08 03:12:37.679623 | PLAY RECAP
2026-04-08 03:12:37.679686 | instance | ok: 2 changed: 2 unreachable: 0 failed: 1 skipped: 0 rescued: 0 ignored: 0
2026-04-08 03:12:37.679716 | 
2026-04-08 03:12:37.781393 | RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/run.yaml@main]
2026-04-08 03:12:37.791661 | POST-RUN START: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/post.yml@main]
2026-04-08 03:12:38.431261 | 
2026-04-08 03:12:38.431416 | PLAY [all]
2026-04-08 03:12:38.446485 | 
2026-04-08 03:12:38.446593 | TASK [gather-host-logs : creating directory for system status]
2026-04-08 03:12:38.775244 | instance | changed
2026-04-08 03:12:38.782164 | 
2026-04-08 03:12:38.782261 | TASK [gather-host-logs : Get logs for each host]
2026-04-08 03:12:39.162228 | instance | + systemd-cgls --full --all --no-pager
2026-04-08 03:12:39.181771 | instance | + ip addr
2026-04-08 03:12:39.184457 | instance | + ip route
2026-04-08 03:12:39.186092 | instance | + lsblk
2026-04-08 03:12:39.191581 | instance | + mount
2026-04-08 03:12:39.194697 | instance | + docker images
2026-04-08 03:12:39.216015 | instance | + brctl show
2026-04-08 03:12:39.216701 | instance | /bin/bash: line 8: brctl: command not found
2026-04-08 03:12:39.216992 | instance | + ps aux --sort=-%mem
2026-04-08 03:12:39.238823 | instance | + dpkg -l
2026-04-08 03:12:39.250841 | instance | + CONTAINERS=($(docker ps -a --format '{{ .Names }}' --filter label=zuul))
2026-04-08 03:12:39.251564 | instance | ++ docker ps -a --format '{{ .Names }}' --filter label=zuul
2026-04-08 03:12:39.271215 | instance | + '[' '!' -z '' ']'
2026-04-08 03:12:39.324090 | instance | ok: Runtime: 0:00:00.113868
2026-04-08 03:12:39.331943 | 
2026-04-08 03:12:39.332024 | TASK [gather-host-logs : Downloads logs to executor]
2026-04-08 03:12:39.945368 | instance | changed:
2026-04-08 03:12:39.945516 | instance | created directory /var/lib/zuul/builds/0f45b49b9fed45abac46197190a5b9cd/work/logs/instance
2026-04-08 03:12:39.945544 | instance | cd+++++++++ system/
2026-04-08 03:12:39.945566 | instance | >f+++++++++ system/brctl-show.txt
2026-04-08 03:12:39.945587 | instance | >f+++++++++ system/docker-images.txt
2026-04-08 03:12:39.945606 | instance | >f+++++++++ system/ip-addr.txt
2026-04-08 03:12:39.945628 | instance | >f+++++++++ system/ip-route.txt
2026-04-08 03:12:39.945650 | instance | >f+++++++++ system/lsblk.txt
2026-04-08 03:12:39.945671 | instance | >f+++++++++ system/mount.txt
2026-04-08 03:12:39.945690 | instance | >f+++++++++ system/packages.txt
2026-04-08 03:12:39.945709 | instance | >f+++++++++ system/ps.txt
2026-04-08 03:12:39.945730 | instance | >f+++++++++ system/systemd-cgls.txt
2026-04-08 03:12:39.954812 | 
2026-04-08 03:12:39.954880 | LOOP [helm-release-status : creating directory for helm release status]
2026-04-08 03:12:40.151995 | instance | changed: "values"
2026-04-08 03:12:40.321231 | instance | changed: "releases"
2026-04-08 03:12:40.340514 | 
2026-04-08 03:12:40.340660 | TASK [helm-release-status : Gather get release status for helm charts]
2026-04-08 03:12:40.541022 | instance | /bin/bash: line 3: kubectl: command not found
2026-04-08 03:12:40.877723 | instance | ok: Runtime: 0:00:00.007133
2026-04-08 03:12:40.884287 | 
2026-04-08 03:12:40.884356 | TASK [helm-release-status : Downloads logs to executor]
2026-04-08 03:12:41.365023 | instance | changed:
2026-04-08 03:12:41.365296 | instance | cd+++++++++ helm/
2026-04-08 03:12:41.365357 | instance | cd+++++++++ helm/releases/
2026-04-08 03:12:41.365403 | instance | cd+++++++++ helm/values/
2026-04-08 03:12:41.377832 | 
2026-04-08 03:12:41.377923 | TASK [describe-kubernetes-objects : creating directory for cluster scoped objects]
2026-04-08 03:12:41.575372 | instance | changed
2026-04-08 03:12:41.607353 | 
2026-04-08 03:12:41.607485 | TASK [describe-kubernetes-objects : Gathering descriptions for cluster scoped objects]
2026-04-08 03:12:41.838392 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
2026-04-08 03:12:41.838549 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
2026-04-08 03:12:41.844249 | instance | environment: line 1: kubectl: command not found
2026-04-08 03:12:41.845536 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value
2026-04-08 03:12:41.846869 | instance | environment: line 1: kubectl: command not found
2026-04-08 03:12:41.848401 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value
2026-04-08 03:12:41.849196 | instance | environment: line 1: kubectl: command not found
2026-04-08 03:12:41.850144 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value
2026-04-08 03:12:41.852698 | instance | environment: line 1: kubectl: command not found
2026-04-08 03:12:41.853896 | instance | environment: line 1: kubectl: command not found
2026-04-08 03:12:41.853916 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value
2026-04-08 03:12:41.855511 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value
2026-04-08 03:12:42.140818 | instance | ok: Runtime: 0:00:00.027292
2026-04-08 03:12:42.146968 | 
2026-04-08 03:12:42.147030 | TASK [describe-kubernetes-objects : creating directory for namespace scoped objects]
2026-04-08 03:12:42.338711 | instance | changed
2026-04-08 03:12:42.344353 | 
2026-04-08 03:12:42.344424 | TASK [describe-kubernetes-objects : Gathering descriptions for namespace scoped objects]
2026-04-08 03:12:42.572881 | instance | environment: line 5: kubectl: command not found
2026-04-08 03:12:42.573426 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
2026-04-08 03:12:42.573718 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
2026-04-08 03:12:42.574110 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
2026-04-08 03:12:42.880419 | instance | ok: Runtime: 0:00:00.009552
2026-04-08 03:12:42.887482 | 
2026-04-08 03:12:42.887551 | TASK [describe-kubernetes-objects : Downloads logs to executor]
2026-04-08 03:12:43.398096 | instance | changed:
2026-04-08 03:12:43.398346 | instance | cd+++++++++ objects/
2026-04-08 03:12:43.398387 | instance | cd+++++++++ objects/cluster/
2026-04-08 03:12:43.398417 | instance | cd+++++++++ objects/namespaced/
2026-04-08 03:12:43.410129 | 
2026-04-08 03:12:43.410229 | TASK [gather-pod-logs : creating directory for pod logs]
2026-04-08 03:12:43.606246 | instance | changed
2026-04-08 03:12:43.612951 | 
2026-04-08 03:12:43.613062 | TASK [gather-pod-logs : creating directory for failed pod logs]
2026-04-08 03:12:43.855899 | instance | changed
2026-04-08 03:12:43.860991 | 
2026-04-08 03:12:43.861083 | TASK [gather-pod-logs : retrieve all kubernetes logs, current and previous (if they exist)]
2026-04-08 03:12:44.083711 | instance | environment: line 3: kubectl: command not found
2026-04-08 03:12:44.397732 | instance | ok: Runtime: 0:00:00.008904
2026-04-08 03:12:44.405239 | 
2026-04-08 03:12:44.405328 | TASK [gather-pod-logs : Downloads pod logs to executor]
2026-04-08 03:12:44.882179 | instance | changed:
2026-04-08 03:12:44.882360 | instance | cd+++++++++ pod-logs/
2026-04-08 03:12:44.882399 | instance | cd+++++++++ pod-logs/failed-pods/
2026-04-08 03:12:44.892946 | 
2026-04-08 03:12:44.893026 | TASK [gather-prom-metrics : creating directory for helm release descriptions]
2026-04-08 03:12:45.089622 | instance | changed
2026-04-08 03:12:45.095288 | 
2026-04-08 03:12:45.095357 | TASK [gather-prom-metrics : Get metrics from exporter services in all namespaces]
2026-04-08 03:12:45.304167 | instance | /bin/bash: line 2: kubectl: command not found
2026-04-08 03:12:45.633959 | instance | ok: Runtime: 0:00:00.034469
2026-04-08 03:12:45.639869 | 
2026-04-08 03:12:45.639935 | TASK [gather-prom-metrics : Get ceph metrics from ceph-mgr]
2026-04-08 03:12:45.845505 | instance | /bin/bash: line 2: kubectl: command not found
2026-04-08 03:12:45.875207 | instance | ceph-mgr endpoints:
2026-04-08 03:12:46.176176 | instance | ok: Runtime: 0:00:00.037333
2026-04-08 03:12:46.183453 | 
2026-04-08 03:12:46.183556 | TASK [gather-prom-metrics : Get metrics from fluentd pods]
2026-04-08 03:12:46.383556 | instance | /bin/bash: line 4: kubectl: command not found
2026-04-08 03:12:46.720233 | instance | ok: Runtime: 0:00:00.040975
2026-04-08 03:12:46.725364 | 
2026-04-08 03:12:46.725434 | TASK [gather-prom-metrics : Downloads logs to executor]
2026-04-08 03:12:47.199971 | instance | changed: cd+++++++++ prometheus/
2026-04-08 03:12:47.208676 | 
2026-04-08 03:12:47.208738 | TASK [gather-selenium-data : creating directory for helm release descriptions]
2026-04-08 03:12:47.462465 | instance | changed
2026-04-08 03:12:47.468176 | 
2026-04-08 03:12:47.468250 | TASK [gather-selenium-data : Get selenium data]
2026-04-08 03:12:47.680730 | instance | + cp '/tmp/artifacts/*' /tmp/logs/selenium/.
2026-04-08 03:12:47.682240 | instance | cp: cannot stat '/tmp/artifacts/*': No such file or directory
2026-04-08 03:12:48.003666 | instance | ERROR
2026-04-08 03:12:48.003908 | instance | {
2026-04-08 03:12:48.003955 | instance |   "delta": "0:00:00.006176",
2026-04-08 03:12:48.003984 | instance |   "end": "2026-04-08 03:12:47.682595",
2026-04-08 03:12:48.004013 | instance |   "msg": "non-zero return code",
2026-04-08 03:12:48.004039 | instance |   "rc": 1,
2026-04-08 03:12:48.004064 | instance |   "start": "2026-04-08 03:12:47.676419"
2026-04-08 03:12:48.004089 | instance | }
2026-04-08 03:12:48.004121 | instance | ERROR: Ignoring Errors
2026-04-08 03:12:48.010824 | 
2026-04-08 03:12:48.010909 | TASK [gather-selenium-data : Downloads logs to executor]
2026-04-08 03:12:48.477470 | instance | changed: cd+++++++++ selenium/
2026-04-08 03:12:48.485064 | 
2026-04-08 03:12:48.485133 | PLAY RECAP
2026-04-08 03:12:48.485185 | instance | ok: 23 changed: 23 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 1
2026-04-08 03:12:48.485206 | 
2026-04-08 03:12:48.590463 | POST-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/post.yml@main]
2026-04-08 03:12:48.602352 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main]
2026-04-08 03:12:49.214086 | 
2026-04-08 03:12:49.214197 | PLAY [all]
2026-04-08 03:12:49.224935 | 
2026-04-08 03:12:49.225033 | TASK [fetch-output : Set log path for multiple nodes]
2026-04-08 03:12:49.269637 | instance | skipping: Conditional result was False
2026-04-08 03:12:49.280730 | 
2026-04-08 03:12:49.280820 | TASK [fetch-output : Set log path for single node]
2026-04-08 03:12:49.325889 | instance | ok
2026-04-08 03:12:49.332263 | 
2026-04-08 03:12:49.332355 | LOOP [fetch-output : Ensure local output dirs]
2026-04-08 03:12:49.702847 | instance -> localhost | ok: "/var/lib/zuul/builds/0f45b49b9fed45abac46197190a5b9cd/work/logs"
2026-04-08 03:12:49.898382 | instance -> localhost | changed: "/var/lib/zuul/builds/0f45b49b9fed45abac46197190a5b9cd/work/artifacts"
2026-04-08 03:12:50.100066 | instance -> localhost | changed: "/var/lib/zuul/builds/0f45b49b9fed45abac46197190a5b9cd/work/docs"
2026-04-08 03:12:50.116010 | 
2026-04-08 03:12:50.116123 | LOOP [fetch-output : Collect logs, artifacts and docs]
2026-04-08 03:12:50.710726 | instance | changed: .d..t...... ./
2026-04-08 03:12:50.710948 | instance | changed: All items complete
2026-04-08 03:12:50.710982 | 
2026-04-08 03:12:51.140613 | instance | changed: .d..t...... ./
2026-04-08 03:12:51.582789 | instance | changed: .d..t...... ./
2026-04-08 03:12:51.599301 | 
2026-04-08 03:12:51.599412 | LOOP [merge-output-to-logs : Move artifacts and docs to logs dir]
2026-04-08 03:12:51.991336 | instance -> localhost | ok: Item: artifacts Runtime: 0:00:00.007541
2026-04-08 03:12:52.223757 | instance -> localhost | ok: Item: docs Runtime: 0:00:00.008546
2026-04-08 03:12:52.236882 | 
2026-04-08 03:12:52.237021 | PLAY [all]
2026-04-08 03:12:52.244241 | 
2026-04-08 03:12:52.244303 | TASK [remove-build-sshkey : Remove the build SSH key from all nodes]
2026-04-08 03:12:52.661380 | instance | changed
2026-04-08 03:12:52.669415 | 
2026-04-08 03:12:52.669466 | PLAY RECAP
2026-04-08 03:12:52.669515 | instance | ok: 5 changed: 4 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0
2026-04-08 03:12:52.669536 | 
2026-04-08 03:12:52.833634 | POST-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main]
2026-04-08 03:12:52.845970 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post-logs.yaml@main]
2026-04-08 03:12:53.418846 | 
2026-04-08 03:12:53.418958 | PLAY [localhost]
2026-04-08 03:12:53.428117 | 
2026-04-08 03:12:53.428186 | TASK [Generate Zuul manifest]
2026-04-08 03:12:53.444789 | localhost | ok
2026-04-08 03:12:53.457925 | 
2026-04-08 03:12:53.458000 | TASK [generate-zuul-manifest : Generate Zuul manifest]
2026-04-08 03:12:53.794380 | localhost | changed
2026-04-08 03:12:53.807750 | 
2026-04-08 03:12:53.807846 | TASK [generate-zuul-manifest : Return Zuul manifest URL to Zuul]
2026-04-08 03:12:53.838442 | localhost | ok
2026-04-08 03:12:53.848023 | 
2026-04-08 03:12:53.848085 | TASK [Upload logs]
2026-04-08 03:12:53.868691 | localhost | ok
2026-04-08 03:12:53.964708 | 
2026-04-08 03:12:53.964828 | TASK [Set zuul-log-path fact]
2026-04-08 03:12:53.984292 | localhost | ok
2026-04-08 03:12:53.999703 | 
2026-04-08 03:12:53.999813 | TASK [set-zuul-log-path-fact : Set log path for a build]
2026-04-08 03:12:54.032602 | localhost | ok
2026-04-08 03:12:54.041669 | 
2026-04-08 03:12:54.041796 | TASK [upload-logs : Create log directories]
2026-04-08 03:12:54.377112 | localhost | changed
2026-04-08 03:12:54.383175 | 
2026-04-08 03:12:54.383267 | TASK [upload-logs : Ensure logs are readable before uploading]
2026-04-08 03:12:54.739254 | localhost -> localhost | ok: Runtime: 0:00:00.005047
2026-04-08 03:12:54.746667 | 
2026-04-08 03:12:54.746764 | TASK [upload-logs : Upload logs to log server]
2026-04-08 03:12:55.158778 | localhost | Output suppressed because no_log was given
2026-04-08 03:12:55.162306 | 
2026-04-08 03:12:55.162370 | LOOP [upload-logs : Compress console log and json output]
2026-04-08 03:12:55.202748 | localhost | skipping: Conditional result was False
2026-04-08 03:12:55.210235 | localhost | skipping: Conditional result was False
2026-04-08 03:12:55.226953 | 
2026-04-08 03:12:55.227129 | LOOP [upload-logs : Upload compressed console log and json output]
2026-04-08 03:12:55.269857 | localhost | skipping: Conditional result was False
2026-04-08 03:12:55.270233 | 
2026-04-08 03:12:55.274319 | localhost | skipping: Conditional result was False
2026-04-08 03:12:55.289832 | 
2026-04-08 03:12:55.289960 | LOOP [upload-logs : Upload console log and json output]