The Promise of Cloud

Part 1 of AWS Series: IO Characteristics of Elastic Block Storage(EBS) in AWS: Grades are out

Part 2 of AWS Series: Migrating Oracle Applications into AWS

Delphix Version 4.1 supports deployment into AWS and so I have been evaluating AWS for the past couple of months. As I speak to partners and customers, it is clear that moving into an IaaS model is not a question of 'if' but 'when'. As organizations realize the full cost of their current investments in on-premise infrastructure, moving to a cost-efficient Iaas model becomes increasingly attractive.

The key questions I found interesting and would like to delve into in this blog are:

1. How easy is it to migrate applications and data into the cloud?

2. How do you realize all the benefits that a cloud infrastructure promises?

I tackled the first question in a recent blog about my experience of migrating my test/dev environments into AWS using Delphix Replication. Migrating large databases and complex enterprise applications to the cloud using traditional methods is prohibitively complex and requires tiptoeing through several manual steps.

For example, to migrate an Oracle E-Business Suite application to the cloud, the Application Tier and the Database have to be moved separately and manually stitched back together. This will not scale to an enterprise level, and is unnecessarily complicated even for a handful of environments. Using Delphix's AppData Engine, all the environments can be 'Replicated' and brought up in the cloud with very little effort and no downtime.

Promise of Cloud

After migrating applications to the cloud, how do organizations then realize its full potential? I would submit that "Operational Agility" and "Cost Savings" are two of the top benefits of an IaaS model that most organizations would like to realize. After running my experiments in AWS for a couple of months, I can clearly see the benefits. Scaling resources up and down is easy and you pay for only what you need and when you need it.  But Data is still a constraint,  involves recurring costs, and is difficult to move even within the cloud.

Unlock the Agility in Data

Here are the steps to create a copy of a database application, say size 1TB, in AWS:

1. Shutdown the DB

2. Create snapshots of all the volumes attached

3. Create new volumes from the snapshots

4. Attach new volumes to new compute nodes

5. Bring up and stitch the database with the new volumes

This process is complex, time-consuming and requires downtime. The compute portion of an application comes up fast, but the data always lags behind. For database applications this means multiple hours of downtime or sub-optimal performance and lost productivity. Even in an IaaS model that promises rapid scaling of resources, data becomes an anchor that slows down productivity and prevents organizations from realizing the full benefits of their cloud infrastructure.

Using Delphix, creating an environment does not involve adding storage, nor does it involve any data movement - Delphix handles storage management transparently. Once the compute node is available, the application can be provisioned onto it within a few minutes. Delphix enables rapid scaling and teardown of database applications along with the rest of the infrastructure. Data is no longer an anchor and organizations can now realize the full benefits of their IaaS infrastructure. Delphix unlocks the constraints around data, helps organizations realize the full potential of their cloud deployment.

Storage Cost Savings

Figure 1: Storage Footprint and Cost multiplies w/ Environments. Delphix curtails this cost explosion

Storage is typically one of the biggest contributors to the TCO of data in cloud. For organizations that rely on database applications, storage cost can be as high as 60%-70% of total cloud expenditure. Delphix uses two key technologies to minimize the storage footprint of environments. Delphix thin-provisions databases, ensuring storage footprint does not scale with environments. Delphix also compresses the data on disk, typically achieving 3:1 compression ratio.

For example, 10 environments of a 1TB database would require 10TB of storage without Delphix. Using Delphix, the storage footprint will be 667GB - a 14x reduction. Without Delphix, storage TCO increases linearly with the number of environments and storage required per environment. With Delphix, there is 1 shared pool managed by Delphix, as opposed to multiple underutilized storage volumes managed by the user.The chart in Figure 1 illustrates this. The benefits do not stop with cost savings alone: Delphix simplifies storage management. With Delphix, there is 1 shared pool managed by Delphix, as opposed to 20 underutilized storage volumes managed by the user.

Delphix simplifies storage management in cloud; Storage savings from Delphix amplify with total footprint.

Rigid SLAs

In my previous blog I talked about  IO Characteristics of EBS storage options in AWS. I highlighted the importance of allocating storage with "Provisioned IOPS (PIOPS)" to ensure adequate response times for database applications. For a typical test/dev environment with 1TB footprint, AWS recommends provisioning up to 5000 IOPS [reference to AWS whitepaper].

A project that needs 10 such environments will require 50000 IOPS. Rigidly allocating IOPS introduces inefficiency of usage as load varies on each environment through the project lifecycle. At $0.19 per GB/month Provisioned IOPS are one of the more expensive resources to purchase in AWS. Today an environment like above, 1TB w/ 5000 IOPS would cost around $500 per month. If we need 10 such environments, that's a yearly cost of $60,000.

In order to be cost-efficient in using such resources, allocation needs to be carefully orchestrated across the environments through the project life cycle. Dev environments may require more IOPS during the initial phases of projects, while Test/Staging environments may require more during release cycles. Though it is easy to change IOPS for a single instance, the process becomes unnecessarily complex and difficult to manage over large deployments.

Elastic SLAs

Delphix converts provisioned IOPS into what I would like to call "Elastic IOPS". Using this model, you do not need to orchestrate IOPS across environments. When  environments are virtualized through Delphix, all the IO is serviced by Delphix. A single shared pool of IOPS is purchased for Delphix, that supports all the environments. No need to pre-commit SLAs for different environments. By creating a single shared pool, IOPS are efficiently allocated across the environments based on needs through the project lifecycle. Delphix simplifies management and allocation of SLAs across your environments

Cost Savings for IOPS

Using Delphix, not only do a set of environments use a single manageable pool of IOPS, they will only need a fraction of the total requirement. This reduction in IOPS is facilitated by our cache.

Delphix Caching

One unique feature of the cache tier is in-memory virtualization. Delphix caches only one copy of any block in memory regardless of the number of environments reading it. This technology allows Delphix to service the working set of all the environments using only a fraction of the memory footprint. In order to match this performance, traditional environments need N times the memory.

Figure 2: OLTP Workloads see benefits scale with IOPS per Environment

OLTP

For an OLTP application virtualized through Delphix, I typically see ~75% hit rate in the cache. That means, if the application needs 5000 IOPS, I only need 1250 IOPS from the storage, as the rest of the IOPS are serviced by the Delphix Cache. At scale, if a project needs 10 such environments, Delphix will only require 12500 IOPS instead of 50,000. That is a 4x reduction in cost of the IOPS that a project will need.

Figure 3: Analytics Workloads will see benefits scale with IOPS per Environment

Analytics For applications with Analytics or reporting style workloads, the benefits of the cache are even more pronounced - I typically see 90-95% hit rate. Delphix delivers almost 10x savings in total IOPS required.

Delphix multiplies the value of purchased SLAs by 4x - 10x.

Conclusion

The promise of cloud is better operational agility at lower TCO of infrastructure. But for database applications, data is still a constraint - forming an anchor that drags down the agility benefits that cloud promises offer. The cost of storage and cost of SLAs around storage make it difficult to realize the full potential of cloud for most organizations. Delphix helps remove these barriers by minimizing storage footprint, simplifying storage management and multiplying the value of SLAs.

Acknowledgements: Thanks to our Andrew for putting together the charts and Louis for editorial help.