After many successful years of operation, at last we decided to do something about our aging VM management software and its underlying server hardware. The custom built software solution - once the pride of the nine engineers - was falling behind what is required to offer competitive VM based managed services to our customers. Additionally components like hard disks were starting to show serious signs of age. You can read more about this in a separate blog post.
After careful consideration we chose Nutanix as our in house infrastructure instead of moving our VM's in the public cloud. Everything looked promising, but we were soon faced with a tough software development challenge. Beyond the challenging task to replace the custom built software with a Terraform and a GitOps based workflow, we also had the challenging task of finding a way that Puppet, our automation software, could gather all It’s required information within this new ecosystem.
When Puppet sets up a client’s managed service and the supporting tech stack, for example monitoring, it needs to know which client the service belongs to as well as a few other important parameters. In the past, this was solved by running a Cron job triggering a script that wrote a static file into the filesystem of the VM. This was working acceptably and the same concept was even extended to support AWS based instances.
We had a few ideas on how to solve this problem without touching the existing solution, for example using cloud-init or a direct connection from Puppet to the Nutanix appliance via API - however, all of them were ruled out due to serious downsides. After further deliberation, as well as a few sketches on the whiteboard we came up with Meko (metadata Kollektor -German for the collector).
It also makes use of caching so it will also work if the provider can not provide any data.
Meko is used by Facter which is built into Puppet and allows the use of external data inside Puppet. We already use Facter to load file based data, so we only had to configure it to communicate to the REST API using HTTPS. Puppet knows which infrastructure it runs on so it was easy to keep the detection part in Facter and to tell MEKO which backend to use to resolve the data.
Meko and the required Puppet changes were rolled out into production without any issues:
We are very satisfied that this worked out so well. We look forward to rolling out Nutanix into our second Datacenter, as well as it’s used for our unmanaged servers solution.
Never miss an update again!