Delphix Replication

Customers use Delphix to efficiently deliver production data to application development and QA environments.

Customers use Delphix to efficiently deliver production data to application development and QA environments. As Delphix becomes an integral part of the application development life cycle, protecting this data against catastrophic failure becomes critical. Delphix provides a native service to replicate application data from one Delphix engine (source) to another (target). Application data can be stored efficiently, with block aware compression giving 2-4x savings in storage. Fast and efficient recovery can be done at a finer granularity compared to any Storage or VM based solution. For example, we can recover data from a single database or from a table within a database.

replication in delphix 3.0

Today our customers use replication to service different use cases. Replication can be used to provide failover protection to critical application development environments. Replicating to a local standby Delphix engine provides protection against server or storage crashes. Replicating to a remote Delphix engine provides protection against site outages.  Replication can also be used as a migration tool for infrastructure expansion, load balancing and storage migration. We have  customers using Delphix as their primary data protection solution for application development environments.

In our latest release, 3.1, we improved replication facilities to provide better value to our customers. Majority of these improvements came from using our in-house Delphix Session Protocol (DSP) developed by my colleague Peng Dai. Using DSP as the primary transport for our native Replication service enabled a number of enhancements to performance and functionality. In this post I will touch on some of them. A thorough discussion of DSP is out of the scope of this post and Peng will be talking about that in his upcoming blog post.

Improved Performance

replication performance enhancements

The chart in Fig:2 shows replication throughput between two Delphix engines. The Delphix engine in this case synchronizes with three databases of different sizes, each supporting 12 Virtual Databases (VDBs). These VDBs handle read heavy, write heavy and OLTP style workloads from their respective applications. The Delphix engine is replicated with all the databases synchronized and running their workloads. On our test platform, we can see more than 3x improvement in throughput achieved between Delphix 3.0 and 3.1. Note that this improvement is currently tied to the underlying physical infrastructure available for Replication. For our upcoming release, we are investing in reducing the dependency on physical infrastructure through the use of better compression and throughput throttling. The details of these will be covered in an upcoming blog post.

Transport Enhancements

With 3.1, we migrated the Replication transport from NDMP to DSP. DSP enables efficient use of network bandwidth and the ability to recover gracefully from transport errors. In 3.1, DSP allows us to overcome transport level outages by automatically resuming replication. This is especially useful for transferring large environments across a WAN.

Interoperability

Another major improvement to Replication in 3.1 is the ability to use the replication target as another full-fledged Delphix engine. Until now, the replication target was only used as a standby for another Delphix engine, providing HA/DR. With 3.1, customers can use the replication target to synchronize with other production databases and provision VDBs, while still replicating other environments. This will open more use cases and increase the value customers can draw from the second Delphix engine.

replication in delphix 3.1

Enhanced Topologies

In 3.1, customers can configure complex topologies for replication. A Delphix engine can replicate different environments to multiple replication target engines. One replication target can receive replication streams from multiple primary engines. This enables customers to distribute replication services across different engines to get maximum protection with minimal overhead. Replication policies can then be orchestrated to suit the needs of the different application development environments.

With the offerings in our latest release, you can imagine a scenario where multiple Delphix engines are cross-purposed in a complex topology, to act as an extended protection layer with superior recovery time and granularity of recovery points as well as providing high availability of multiple application development environments.

Given the vantage point we occupy in our customers’ application development environment, Delphix becomes a natural and superior alternative to supplant backup/recovery services provided by other VM or storage based solutions. The improvements in speed and functionality in our latest release are just the opening salvo in a series of enhancements we are planning to roll out in our upcoming releases.