Automated Delphix Deployments

Why Automation?

Managing a growing number of instances of any software is a problem for anyone who interacts with those components. Recent trends in automation have allowed IT teams to streamline deployments, as automation tools like Selenium and Chef optimize QA testing and VM configuration, respectively.

This blog explores the automation of Delphix, which I’ve broken into 3 main categories:

1. VMware Configuration + Virtual Machine Deployment

2. Delphix Engine Base Configuration

3. Creation and Management of Data

All of these steps can be automated using industry standard tools and our API. As we further our developer-centric, open source approach, I imagine we will fine tune our automation capabilities. Until then, check out what I’ve discovered is possible using Terraform, Python, and Delphix APIs!

The Story of Infrastructure as Code

Software companies have been building tools to help professionals manage increasingly complex IT infrastructure. One example of this is Hashicorp’s Terraform, which is intended to help these professionals “build, change, and version infrastructure safely and efficiently.” Terraform promotes the idea of “infrastructure as code”. This means that all parts of your infrastructure are managed through Terraform .tf files, which allow you to control your servers and VMs programmatically rather than manually. For example, rather than manually creating and configuring dozens of AWS EC2 instances, you can manage every single one through a file, with each instance defined in a ‘resource’ blob. A single command will initiate the creation of these instances, and any time you want to change the settings of any instance, you can edit the text file and run Terraform again. Terraform will programmatically change the settings you modified and your work is complete.

Example .tf Terraform file for Delphix VM deployment in vSphere.

Delphix VM Management: vSphere

Today, most Delphix customers manage their VMs through vSphere, a client that lets you manage all your VMs and ESX hosts. Fortunately, Hashicorp has developed a Terraform provider for vSphere that allows you to configure vSphere objects with the infrastructure as code mentality. Just like in my AWS example above, I will show you how you can use the Terraform / vSphere provider to create and manage your Delphix VMs programmatically. Feel free to take a look at their open source GitHub repo if you are curious.

Here is a list of the providers that Hashicorp has built Terraform integrations for.

Without Terraform and the gift of automation, Delphix administrators usually manage their VMs in an interface like the one below. Each VM would be individually created in a wizard that walks you through the various steps of VM configuration. If you’re only deploying a handful of engines, it’s not a problem, but once you get to the stage where you’re deploying at scale, this process needs to be fully automated.

Example vSphere Web Client with Delphix VMs provisioned.

Terraform vSphere Provider

Accomplishing VM setup with Terraform removes the manual configuration steps for each VM. To begin, you will have to set up a single OVA to be used as a template. All you have to do is create one Delphix VM, then convert it into a VM template using the option shown below. This template will be used as your primary object from which your other VMs will be automatically created using Terraform. This is one manual step that is required so that Terraform has a template to reference for the VMs it will create.

Converting a provisioned Delphix VM into a Template to use with Terraform.

Once your template has been created, now we get to dig into Terraform to start deploying infrastructure as code.

Without going too deep into the Terraform vSphere provider syntax, which you can learn from their documentation, let me show you what Terraform does to automate creation and management of VMs that it creates. By initiating Terraform in your Terraform working directory, you can see the changes the Terraform will make on your vSphere instance by entering the commands `terraform plan` and then `terraform apply`.

You can see that Terraform detects certain vSphere VM attributes and outputs what it plans to set each attribute as based on the terraform .tf files you have written.

Terraform apply example showing a few of the attributes that Terraform sets during VM creation. The full list of properties is not captured in this screenshot.​​​​​​

Here, you can see that VMware VMs have certain properties like ‘id’ and objects like ‘disks’. You can add disks to VMs through the vSphere provider, but we can do it through Terraform as well.

One resource of type ‘vsphere_virtual_machine’ named ‘terraform-delphix-vm’ with properties like name and number of CPUs defined in a Terraform file. The variables in this example are saved in another file which can be configured to your liking.

With this output, you can confirm the properties have been properly set, and confirm the changes by entering a simple `yes`.

Confirmation of your plan to add 1 object to vSphere, just enter `yes`.

Terraform will begin issuing these changes onto your vSphere client, and you’re good to go! In the vSphere client, you should see a new job running that shows the new object being created.

Conclusion

Managing a large number of software doesn’t need to be difficult. With Terraform, you can configure as many Delphix VMs as needed in a simple, code-based approach. In the next installment of this blog, see how you can use python to programmatically interface with Delphix VDBs, and an alpha version of our own Terraform / Delphix provider!