Who is this article aimed at?
The project is useful for service providers or operators of small private clouds, who previously built crutches with bash scripts and a lot of not so obvious solutions.
Idea
What is the most resilient parasite? Bacterium? Virus? Intestinal worm? Idea. It is tenacious and extremely contagious. Once an idea takes hold of the brain, it is almost impossible to get rid of it. I mean a formed idea, fully conscious, settled in the head.
It all started with jokes from a friend: “Where are the SSH keys?” in the context of the cloud. It would seem that this is a common feature for a provider, but there were no suitable implementations on the horizon so as not to affect user-data with an emphasis on the elementary implementation.
My gaze fell on a part of the OpenStack element called Vendor Data (or, to be more precise, the Nova metadata service, which covers a number of tasks from the point of view of the cloud operator), and, as it turns out, not in vain.
Vendor Data provides an easy way to provide additional information specific to the instance, for example, this could be information for the configuration of pre-installed software or information about which data center and on what basis the instance is deployed to apply any tweaks.
Closer to the point: the Consul KV key-value database was chosen as the most convenient way to store configurations.
According to documentationThe DynamicJSON provider expects the vendor data provider to return the information in the form of a valid JSON object, however, after a short conversation with the developers, the thesis was put forward that the contents escaped with quotes are also valid JSON.
This is not considered expected behavior, based on the OpenStack documentation, and the community has tried to find a solution several times, returning to this issue every few years.
We wrap everything in the rest api provider
After receiving the working version, it was decided to roll out the basic implementation to open source project with the Vendor Data providersince an open source project has a greater chance of development by the community.
About the provider:
-
Based on FastAPI with a foundation for future features;
-
Along the way
/ocdv
mounted Flask-based application that works directly with Consul KV; -
Connected for Flask app keystonemiddleware – this is the official middleware for WSGI-based applications for authorizing incoming requests (in short: the X-Auth-Token header is checked);
-
Along the way
/ocdv/cloud-init
the application expects a POST request with data, including the instance ID. -
Checks for the presence of a key in Consul KV along the way
cloud/instances/${instance_id}/cloud-config
. If there is such a key, then a serialized string with escaped characters is returned (in fact, the standard package is usedjson
) to match the understanding of valid JSON (yes, in one line) and returns a response; -
DynamicJSON parses the response (under the hood the same package
json
) and also returns a response further down the chain to collect results from all providers.
Important: in Vendor Data, the data returned from the provider #cloud-config
was exactly in the key cloud-init
in which cloud-init
and expects to see the configuration, needs to be configured nova-api
in the following way:
[api]
vendordata_providers=DynamicJSON
[email protected]:8000/ocdv/cloud-init
Checking inside the instance
The article assumes that you already have a ready-to-use Consul cluster. Let’s assume the UUID of the instance 00000000-1337-1337-1337-000000000000
. Along the way cloud/instances/00000000-1337-1337-1337-000000000000/cloud-config
put the following content:
#cloud-config
hostname: ocdvworks
Let’s look at Vendor Data:
curl http://169.254.169.254/openstack/latest/vendor_data2.json | jq
{
"cloud-init": "#cloud-config\nhostname: ocdvworks"
}
What to do next with this?
Before starting the instance, you need to create #cloud-config
and place it in the above path in Consul KV. When the instance starts, cloud-init will check for the presence of data in vendor_data2.json
and will then process the configuration. An important point: if the user specified the user-data configuration when creating the instance, then the user configuration takes precedence over the provider one.
Conclusion
I hope that the discovered solution will help operators manage their infrastructure more easily, avoiding the already common crutches.
The project is completely under the MIT license, free for any use.
https://github.com/ib-systems/openstack-consul-dynamic-vendordata
I would be grateful for ideas for improvement and contributions to the source code, since the current implementation was formed “at my leisure” in my free time from all the routine and requires polishing.
Acknowledgement and Usage Notice
The editorial team at TechBurst Magazine acknowledges the invaluable contribution of the author of the original article that forms the foundation of our publication. We sincerely appreciate the author’s work. All images in this publication are sourced directly from the original article, where a reference to the author’s profile is provided as well. This publication respects the author’s rights and enhances the visibility of their original work. If there are any concerns or the author wishes to discuss this matter further, we welcome an open dialogue to address potential issues and find an amicable resolution. Feel free to contact us through the ‘Contact Us’ section; the link is available in the website footer.