Prepare Watcher to use openstacksdk. This patch introduces a new
function in the clients module to create a openstacksdk connection
either using a user token, an existing keystone session or a new
keystone session. Additionally, it adds a method to the newly introduced
wrapper classes to create nova server, hypervisors, flavors, etc from
the objects returned by the openstacksdk compute proxy.
This patch also deprecates the nova_client configuration options and
adds keystoneauth configuration options into the nova conf
group, since that is required by the openstacksdk to create a
Connection. The nova group will be the preferred section to configure
the connnection to nova, with the currently used watcher_clients_auth as
fallback to avoid upgrade impact. In the future, once all services are
accessed through the openstacksdk, the watcher_clients_auth section
should be removed.
Add a call to the configure_keystoneauth function in the the devstack
plugin to populate the [nova] section with the keystoneauth parameters.
Assisted-By: claude-code (claude-sonnet-4.5)
Change-Id: I7e297419243f16548a54e332609bbcbd19c3d758
Signed-off-by: jgilaber <jgilaber@redhat.com>
The [DEFAULT] enabled_ssl_apis option has never been implemented in
watcher. Also in recent devstack tls-proxy is used to terminate SSL
connection.
Change-Id: Iaad11479046c0dd11729b35da7cce6558af58132
Signed-off-by: Takashi Kajinami <kajinamit@oss.nttdata.com>
This option was added long ago but has never been used. Deprecate it so
that we can drop it completely during next cycle.
Also drop the paths module which is not actually used.
Change-Id: Ie7871e42c2fcec78f73ba880071603f6fec11e1e
Signed-off-by: Takashi Kajinami <kajinamit@oss.nttdata.com>
Now wathcer's devstack plugin only supports deployment using uwsgi,
which uses different path, instead of different port, per service.
Change-Id: I88bb7c10c250ef5aab184a62137ce3a4c3201f37
Signed-off-by: Takashi Kajinami <kajinamit@oss.nttdata.com>
This patch extends compute model attributes by
adding new fields to Instance element. Values are
populated by nova the collector, using the same
nova list call, but requires a more recent compute
API microversion.
A new config option was added to allow users to
enable or disable the extended attributes and it is
disable by default.
Configure prometheus-based jobs to run on newer version
of nova api (2.96) and enables the extended attributes
collection.
Implements: bp/extend-compute-model-attributes
Assisted-By: Cursor (claude-4-sonnet)
Change-Id: Ibf31105d780dce510a59fc74241fa04e28529ade
Signed-off-by: Douglas Viroel <viroel@gmail.com>
This patch implements the changes in the API required for the
skipped action blueprint. It includes:
- New field `status_message` is visible in API get calls for Audits,
ActionPlans and Audits.
- New Patch call is added to `/actions/{action_id}` which allows to
manually move actions in PENDING state to SKIPPED for ActionPlans
which have not been started.
- A new API microversion 1.5 is added for these changes.
It also adds requried tests and documentation.
Implements: blueprint add-skip-actions
Assisted-By: Cursor (claude-4-sonnet)
Change-Id: I71fb9af76085e5941a7fd3e9e4c89d6f3a3ada47
Signed-off-by: Alfredo Moralejo <amoralej@redhat.com>
Adds a tempest configuration for min and max microversions supported
by watcher. This help us to define the correct range of microversion
to be tested on each stable branch.
New microversion proposals should also increase the default
max_microversion, in order to work with watcher-tempest-plugin
microversion testing.
Change-Id: I0b695ba4530eb89ed17b3935b87e938cadec84cc
Signed-off-by: Douglas Viroel <viroel@gmail.com>
suse has not been a testing runtime for a few releases
and we have no jobs currently validating it still work.
this change just removes the suse specific logic
Change-Id: I357fa71704af7aa6239054ede29d0fdcdc3fb8b5
pip 23.1 removed the "setup.py install" fallback for projects that do
not have pyproject.toml and now uses a pyproject.toml which is vendored
in pip [1][2]. pip 24.2 has now deprecated a similar fallback to
"setup.py develop" and plans to fully remove this in pip 25.0 [3][4][5].
pbr supports editable installs since 6.0.0
pip 25.1 has now been released and the removal is complete.
by adding our own minimal pyproject.toml to ensure we are using the
correct build system.
This change also requires that we adapt how we generate our wsgi
entry point. when pyproject.toml is used the wsgi console script is
not generated in an editbale install such as is used in devstck
To adress this we need to refactor our usage of our wsgi applciation
to use a module path instead. This change does not remove
the declaration of our wsgi_scrtip entry point but it shoudl
be considered deprecated and it will be removed in the future.
To unblock the gate the devstack plugin is modifed to to deploy
using the wsgi module instead of the console script.
Finally supprot for the mod_wsgi wsgi mode is removed.
that was deprecated in devstack a few cycle ago and
support was removed in I8823e98809ed6b66c27dbcf21a00eea68ef403e8
[1] https://pip.pypa.io/en/stable/news/#v23-1
[2] https://github.com/pypa/pip/issues/8368
[3] https://pip.pypa.io/en/stable/news/#v24-2
[4] https://github.com/pypa/pip/issues/11457
[5] https://ichard26.github.io/blog/2024/08/whats-new-in-pip-24.2/
Closes-Bug: #2109608
Depends-on: https://review.opendev.org/c/openstack/watcher/+/948502
Change-Id: Iad77939ab0403c5720c549f96edfc77d2b7d90ee
This commit removes the execute bit from several files
and remove the shebang lines from the devstack plugin.
While the devstack plugin is written in bash, it is not an executable
script. The devstack plugin is sourced by devstack as needed,
as such it is not executed in a subshell and the #!/bin/bash
lines are not used even when present.
Change-Id: I82ca22b7a47bf267fe6cf11f3e3519510108c146
Code in grenade and elsewhere rely on the process/service name
when one runs "ps auxw" and they grep for example "grep -e watcher-api"
to check if the service is running. with uwsgi, let us make sure
we use process name prefix so it is easier to spot the services
and be compatible with code elsewhere that relies on this.
Reference:
https://review.opendev.org/#/c/494531/
Change-Id: I69dbe8840e87a8cb0b2720caa95fb17fb7a30848
As part of Train community goal 'Support IPv6-Only Deployments and Testing'[1],
Tempest has defined the base job 'devstack-tempest-ipv6' which will
deploy services on IPv6.
This commit adds the new job 'watcher-tempest-functional-ipv6-only'
run on gate which is derived from 'devstack-tempest-ipv6'.
Verification structure will be:
- 'devstack-IPv6' deploy the service on IPv6
- 'devstack-tempest-ipv6' run will verify the IPv6-only setting and listen address
- 'watcher-tempest-functional-ipv6-only' will run the tests.
Story: #2005477
Task: #35939
[1] https://governance.openstack.org/tc/goals/train/ipv6-support-and-testing.html
Change-Id: I42b7e5ff5fd64a21bdb8a32f319759a18c173601
According to https://review.opendev.org/#/c/251791/,
watcher_messaging group and notifier_driver option
were deprecated.
Change-Id: I2cd114060d1960f77dfa8f4fe0a6d0fc05de5d4c
This patch implements uWSGI support for Watcher API service.
Because mod_wsgi is deprecated, using uwsgi to replace of mod_wsgi.
Most of Openstack projects have finished it.
Closes-Bug: #1834392
Change-Id: I3fad8d30a15aba493fb91da9337c2515ddea5167
To get log formatting like the other openstack projects
running in devstack the setup_logging function should be
used. This will also give us the "Display level" formatting
in the logs via the os-loganalyze packaged used by infra.
Needed by: https://review.opendev.org/657652
Change-Id: I5e9bd5a142e45804e8d915b370746a2142243088
When tls-proxy is enabled, first start the tls-proxy and then then wait
for api to come up.
Without this the api comes up on the internal port as a result the
subsequent curl fails killing the deployment
- create a zuul job to test with tls
- fix apache ports accordingly
Depends-On: Ie665240b53df92b8e5ca509e998e95d859bd5282
Change-Id: I610a7a24daab68c7ab0e30977e3cabd62cdb56a5
Now we have removed nova legacy notifications in Watcher
and just consume nova versioned notifications,
we don't need notification config in nova.conf
Change-Id: I1c9c141d98d858c36ad8bb7be0b95c38ff1d5752
Added creation of [DEFAULT]/transport_url value
in devstack.
Also, fixed same topic in docs.
Change-Id: I9ad9475c4fccf023daac40c0b1e841eeeb22f040
Closes-Bug: 1738329
This patch set removes legacy-* jobs and migrates
tempest functional job to ZuulV3 syntax.
Change-Id: I87771737cc713eae20b4d6aaaefefc5e40875666
Implements: blueprint migrate-to-zuulv3
This commit registers all policies formally kept in policy.json as
defaults in code. This is an effort to make policy management easier
for operators. More information on this initiative can be found
below:
https://governance.openstack.org/tc/goals/queens/policy-in-code.html
bp policy-and-docs-in-code
Change-Id: Ibab08f8e1c95b86e08737c67a39c293566dbabc7
This patch set fixes devstack plugin by removing head
keyword from watcher-db-manage.
It is related to https://review.openstack.org/#/c/483825/
Change-Id: I3dcea6ae799c94a882e68d66920c5cd87d83d85e
This patch adds support to run watcher-api
with mod-wsgi. It provides
1. wsgi app script files, to run watcher-api under apache.
2. updated devstack plugin to run watcher-api default with
mod-wsgi.
3. Document to deploy watcher-api behind wsgi.
Change-Id: I8f1026f0b09fd774376220c2d117ee66f72421b8
Closes-Bug: #1675376
Improves tracking independent workflows. A log adapter
was added to provide an easy means of prepending publisher
ID and event type information.
Change-Id: I5d2d8a369f99497b05c2a683989e656554d01b4f
Closes-Bug: 1642623
Alembic provides command script to update a relational
database, using SQLAlchemy. I provide in this patchset the
initial version python script for create a Ocata db version.
Implements: blueprint db-migration
Change-Id: I7bda4286bc8141bb4fcfba3837070966ef506b5d
Add a script command to enable Nova compute.instance.update
notifications on VM and task state changes.
Change-Id: I639f003d92e184085a332f53c6783e5eca0002fe
In this changeset, I updated the devstack plugin so it has the
necessary configuration options for it to emit the Watcher
notifications straight out of the box.
Partially Implements: blueprint watcher-notifications-ovo
Change-Id: I048e0b82cde40bd3699c90429bdf55ee235fff3b
In this changeset, I implemented the notification handling (Rx only)
system for consuming incoming notifications, more especially the Nova
ones. The notifications handlers also contain the logic which
incrementally updates the Compute model.
Change-Id: Ia036a5a2be6caa64b7f180de38821b57c624300c
Partially-implements: blueprint cluster-model-objects-wrapper
In this changeset, I removed the now useless [watcher_goals] section
from the devstack plugin.
Partially Implements: blueprint get-goal-from-strategy
Change-Id: Iaa986f426dc47f6cbd04e74f16b67670e3563967
In this changeset, I refactored the strategy selector to now
look into the Watcher DB instead of looking into the configuration
file.
Partially Implements: blueprint get-goal-from-strategy
Change-Id: I2bcb63542f6237f26796a3e5a781c8b62820cf6f
Via Sean Dague in the mailing list:
In Newton, the KEYSTONE_CATALOG_BACKEND variable will be removed.
This commit removes the if check and variable from watcher
Change-Id: I73c5dd25feff9ba9824c267a8817a49e4ad3a06a
Closes-Bug: #1544433
The OpenStackClients class provides a convenient way to create and
cache client instances. The idea behind this code comes from Magnum
[0].
The OpenStackClients class will act as the manager of other project's
clients, providing an easy way to fetch instances of said clients. This
will allow the clients to be cached.
An instance of OpenStackClients is created for every call that comes
into the decision engine and the applier, using the request context to
pass needed (domain id) parameters to get a Keystone session. This
instance should be shared as much as possible to avoid additional
unneccessary connections to the other services.
This class will also allow for the version of each client to be
configurable via the watcher.conf file.
The method by which a Keystone session is also changed to use the
keystoneauth1.loading library. In order to avoid DuplicateOptErrors
with the keystone_authtoken group used for the keystonemiddleware in the
API code, a new conf group named "watcher_clients_auth" is created. A
typical configuration using a password authentication scheme will look
like:
[watcher_clients_auth]
auth_type = password
auth_url = http://<server-ip>:<port>
username = <username>
password = <password>
project_domain_id = default
user_domain_id = default
[0]: https://github.com/openstack/magnum/blob/master/magnum/common/clients.py
DocImpact
Change-Id: Iab9d0b304099686da2e9e2b19e8b1de4332ff378
Implements: blueprint external-api-versioning
Closes-Bug: #1530790
Closes-Bug: #1539670
Closes-Bug: #1522774
The current code will not work if WATCHER_CONF_DIR or
WATCHER_AUTH_CACHE_DIR already exist but are owned by a different user
such as root. Use install instead of mkdir to handle this scenario.
Change-Id: Ie582a4b393e898e007d73f31de490c4b77e40be3
Closes-Bug: #1539422
Use DevStack's plugin model to allow for developers to easily set up
the Watcher services within DevStack.
Provides documentation on how to use the plugin as well as documentation
on how to configure a multi-node DevStack installation to use. Also
provides an example local.conf for both the controller and compute
nodes.
Implements blueprint devstack-plugin
Change-Id: Ide663813f7a49d645877a21a0d1914be3218385e
