Data sync from production to non-production environments

Data syncs between VIP Platform environments facilitate testing and QA of new features and allows teams to accurately reproduce and examine errors in a non-production environment. After a data sync from a production environment to a non-production environment is completed, the target environment will automatically load shared media files that were uploaded to production. This eliminates the need to copy media between environments.

Limitations

• Data syncs can only occur from a production environment to a non-production environment.
• Non-production environments have access to a production environment’s shared media files in read-only mode.
• WordPress multisite environments that have sites created in addition to the main site (ID 1) must create and maintain a config.yml file for the results of a data sync to work as expected.
• Elasticsearch indexes are not included in the data sync process.  

Actions that occur during a data sync

The data sync operation is designed to not impact the production site in any way. The following process occur only on the target environment:

1. The target environment is protected via “maintenance mode”. This prevents access to the site while the data is being restored and manipulated, and ensures no changes can be made (and lost) during this time.
2. The most recent backup from production is imported into the target environment.
3. Occurrences of the production primary domain in the database are replaced with the primary domain for the target environment. For example, if the production site domain is  example.com and the target non-production site domain is example-com-develop.go-vip.net, then all instances of example.com will be replaced with example-com-develop.go-vip.net in the target database after the sync.
4. Protected options are restored.
5. The wp vip data-cleanup datasync CLI command is run for further cleanup.
6. The vip_datasync_cleanup hook can be used for custom cleanup of application data during this step.
7. Jetpack is (re)connected.
8. Caches are flushed.
9. Maintenance mode is deactivated.

Initiate a data sync in the VIP Dashboard

1. Navigate to the VIP Dashboard for an application.
2. Select the non-production target environment for the data sync (e.g., preprod, develop) from the dropdown located at the upper left of the VIP Dashboard.
3. Select “Data” from the navigation menu on the left.
4. Choose the destination environment for the data sync at the top of the dashboard (e.g., “develop”).
5. Select “Copy from Production” under “Data Sync”.
6. Select the “Copy Data” button to initiate the sync.

Initiate a data sync with VIP-CLI

VIP-CLI command: vip sync [options]

1. Target the production environment with the vip sync command to initiate the data sync.
2. A command line prompt will allow the user to select which non-production environment to sync the data to, or to cancel the sync request.

Protected options

The following options that exist on the target environment are backed up before the data sync begins, then restored after the data sync completes. This prevents environment-specific configurations from being shared between environments. It is not possible to extend or modify this list.

"WP Options" = [
	'jetpack_options',
	'jetpack_private_options',
	'vaultpress',
	'vip_jetpack_connection_pilot_heartbeat',
	'wordpress_api_key',
	'vip_search_index_versions',
	'vip_search_global_index_versions',
];

// On multisites
"Site Meta" = [
	'vip_search_global_index_versions',
];

Custom cleanup operations

The vip_datasync_cleanup hook can be used for custom cleanup of application data (e.g., removing production API keys, performing brief data manipulation) after a data sync. On a WordPress multisite, the cleanup hook is run individually on each network site on the environment.

Custom code that uses the vip_datasync_cleanup hook should be added to a file in the /client-mu-plugins directory. Because data syncs move downward from the production environment, the vip_datasync_cleanup code must be present in the branch that deploys to the target environment (e.g., development or staging).

In this example the vip_datasync_cleanup hook is used to delete the my_social_api_token option on the non-production environment:

/**
 * Run some custom cleanup after a migration.
 *
 * @uses vip_datasync_cleanup
 */
function my_action_vip_datasync_cleanup() {
    // Safety first: Don't do anything in 
    // the production environment
    if ( 'production' === VIP_GO_APP_ENVIRONMENT ) {
        return;
    }
 
    delete_option( 'my_social_api_token' );
}
add_action( 'vip_datasync_cleanup', 'my_action_vip_datasync_cleanup' );