Data sync from production to non-production environments

Data syncs between VIP Platform environments facilitate testing and QA of new features and allows teams to accurately reproduce and examine errors in a non-production environment.

Limitations

• Data syncs can only occur from a production environment to a non-production environment.
• WordPress multisite environments that have sites created in addition to the main site (ID 1) must create and maintain a config.yml file for the results of a data sync to work as expected.
• Elasticsearch indexes are not included in the data sync process.  
• Files uploaded to the production environment are made available to non-production environments in read-only mode by UnionFS. After a data sync, media file URLs on the target environment will automatically point to the shared files.

Actions that occur during a data sync

The data sync operation is designed to not impact the production site in any way; all processing happens on the target environment:

1. The target environment is protected via “maintenance mode”. This prevents access to the site while the data is being restored and manipulated, and ensures no changes can be made (and lost) during this time.
2. The most recent backup from production is loaded into the target environment.
3. Occurrences of the primary production domain in the database are replaced with the primary domain for the target environment. For example, if the production site domain is  example.com and the target non-production site domain is example-com-develop.go-vip.net, then all instances of example.com will be replaced with example-com-develop.go-vip.net in the target database after the sync.
4. Protected options are restored.
5. The wp vip data-cleanup datasync CLI command is run for further cleanup.
6. The vip_datasync_cleanup hook can be used for custom cleanup of your application data during this step.
7. Jetpack is (re)connected.
8. Caches are flushed.
9. Maintenance mode is deactivated.

Running a data sync in the VIP Dashboard

Data syncs can be initiated in the VIP Dashboard. Once logged into the VIP Dashboard:

• Navigate to the “Data” link in the left-hand navigation menu.
• Choose the destination environment for the data sync at the top of the dashboard (e.g., “Develop”).
• Click on “Copy from Production”.
• Follow the prompts in the dashboard and click on the “Copy Data” button when ready.

Running a data sync with VIP-CLI

To run the following commands, VIP-CLI must first be installed on the local machine.

Step 1: Find the application name or ID you wish to sync

You can do this with the vip app list command, which will give you back a list of apps. You want the name or the id.

Step 2: Initiate the sync

Run the sync command with the app name, e.g. vip sync --app=my-app, or the app id, e.g. vip sync --app=000. You will be prompted for the environment you wish to sync to, i.e. the target environment. You can get more information about sync command options by using the help parameter, i.e. vip sync --help.

Protected options

The following options are protected during data syncs, meaning they are backed up on the child site before the sync and restored afterward. This is to prevent environment-specific configs from being shared between environments. It is not possible to extend or modify this list.

"WP Options" = [
	'jetpack_options',
	'jetpack_private_options',
	'vaultpress',
	'vip_jetpack_connection_pilot_heartbeat',
	'wordpress_api_key',
	'vip_search_index_versions',
	'vip_search_global_index_versions',
];

// On multisites
"Site Meta" = [
	'vip_search_global_index_versions',
];

Custom cleanup operations

The vip_datasync_cleanup hook can be used for custom cleanup of application data (e.g., removing production API keys, performing brief data manipulation) after a data sync.

In this example the vip_datasync_cleanup hook is used to delete the my_social_api_token option on the non-production environment:

/**
 * Run some custom cleanup after a migration.
 *
 * @uses vip_datasync_cleanup
 */
function my_action_vip_datasync_cleanup() {
    // Safety first: Don't do anything in 
    // the production environment
    if ( 'production' === VIP_GO_APP_ENVIRONMENT ) {
        return;
    }
 
    delete_option( 'my_social_api_token' );
}
add_action( 'vip_datasync_cleanup', 'my_action_vip_datasync_cleanup' );

On a WordPress multisite, the cleanup hook is run individually on each network site on the environment.