From Zero to Hero: A Tale of Two DBs

It was the best of times, it was the worst of times...

As Redditor "cscareerthrowaway567" writes, s/he recently moved across country to take an exciting new job - his/her first job - as a Junior Software Developer. Everything was going perfect, until that first day on the job. While following the instructions given by his/her new team, the newly minted Developer inadvertently destroyed a production database. (Read more here: https://redd.it/6ez8ag)

Whoops! Here is a little excerpt of the firsthand account of what happened:

I was basically given a document detailing how to setup my local development environment. Which involves run a small script to create my own personal DB instance from some test data. After running the command i was supposed to copy the database url/password/username outputted by the command and configure my dev environment to point to that database. Unfortunately instead of copying the values outputted by the tool, i instead for whatever reason used the values the document had.

Unfortunately apparently those values were actually for the production database (why they are documented in the dev setup guide i have no idea). Then from my understanding that the tests add fake data, and clear existing data between test runs which basically cleared all the data from the production database. Honestly i had no idea what i did and it wasn't about 30 or so minutes after did someone actually figure out/realize what i did.

Now, there are many things that contributed to this debacle: insufficient documentation, production passwords stored in plain text, inadequate firewall restrictions, etc., none of which lie at the feet of this poor developer.

But all of those factors are merely symptoms of the real problem: data is heavy.

While the size of data weighs you down, it is actually everything that surrounds the distribution and handling of the data that encumbers your business: backups, restores, masking, refreshes, archives, flashbacks, configuration. And from time-to-time, the weight gets to you, and something like this happens.

At Delphix, we make data weightless and available to all. Safely. We give developers self-service access to access to data. The access to data is defined according to their policies via a simple role-based access control model. Should developers only have access to masked data? No problem. With a few clicks, cscareerthrowaway567 would have had a copy of the database provisioned in his/her environment. And the data would have been ready in mere minutes, even terabytes-worth. S/he could have reset or refreshed their database with just one click.

And since Delphix only ever reads from production, you have insulated production from the impacts of data requests (and all the things that can go wrong). In fact, Delphix has routinely been used to save the day when events like what happened to cscareerthrowaway567. Recently at a Delphix user group in London, I held a four customer panel that consisted of three of the world's largest banks. All three of those banks said that though they use Delphix for application development and testing today, they originally bought Delphix to restore production due to incidents where developers and DBA's had made mistakes on production.

To close out this post, I will leave you with a story about long time Oracle hero and Delphix user Bobby Durrett on how he saved the day with Delphix. In short, a batch job had destroyed ten tables in a 5.4 TB database at US Foods. Bobby leveraged Delphix to create a production clone in under 15 minutes and had the entire incident closed in under 3 hours, all by himself. Below is the link to Bobby's complete play-by-play.

https://www.bobbydurrettdba.com/2014/07/08/used-delphix-to-quickly-recover-ten-production-tables/

Interested in learning about how to protect production data (and keep your job?) Tune in to this webinar later this month. Sign up here: https://pages.delphix.com/Production-Support-Webinar-registration.html

Production Support Webinar

Date: 29 June 2017
Time: 10:00am BST
Speaker: David Ruthven, Senior Solutions Engineer, Delphix