Skip to content

Data sync from production to non-production environments

Data syncs between VIP Platform environments facilitate testing and quality assurance (QA) of new features and allows teams to accurately reproduce and examine errors in a non-production environment. After a data sync from a production environment to a non-production environment is completed, the target environment will automatically load shared media files that were uploaded to production. This eliminates the need to copy media between environments.

Limitations

Prerequisites

Actions that occur during a data sync

The data sync operation is designed not to impact the production environment in any way. The following actions only occur on the target environment:

  1. The target environment is protected via “maintenance mode”. This prevents access to the environment while the data is being restored and manipulated, and ensures that no changes can be made (and lost) during this time.
  2. The most recent database backup of the production environment is imported into the target environment.
  3. Occurrences of the production environment’s primary domain in the database are replaced with the primary domain of the target environment. For example, if the production environment’s primary domain is  example.com, and the target environment’s primary domain is develop.example.com, then all instances of example.com will be replaced with develop.example.com in the target database after the sync. For target environments that do not yet have a primary domain assigned,  example.com will be replaced by the target environment’s convenience domain (e.g. example-com-develop.go-vip.net).
  4. Protected options are restored.
  5. A wp vip data-cleanup datasync CLI command is run for further cleanup.
  6. The vip_datasync_cleanup hook can be used to perform custom cleanup of application data at this stage.
  7. Jetpack is (re)connected.
  8. Caches are flushed.
  9. Maintenance mode is deactivated.

Initiate a data sync

  1. Navigate to the VIP Dashboard.
  2. Select a non-production environment from the environment dropdown located at the upper left of the VIP Dashboard.
  3. Select “Database” from the sidebar navigation at the left of the screen.
  4. Select Data Sync from the submenu.
  5. Select the button labeled “Copy Data” to initiate the sync.

VIP-CLI command: vip sync [options]

To initiate a data sync, run the vip sync command.

Format the vip sync command to target the non-production environment to which the production database should be synced. For example, to sync data from the production environment of the “mystestsite” application to the develop environment:

vip @mytestsite.develop sync

If the vip sync command is run without targeting a specific application or environment, the user will be prompted in the command line to select which non-production environment to sync the data to, or to cancel the sync request.

Protected options

A specific set of options that exist on the target environment are backed up before the data sync begins, then restored after the data sync has completed. This prevents environment-specific configurations from being shared between environments.

The protected options list cannot be extended or modified.

"WP Options" = [
	'jetpack_options',
	'jetpack_private_options',
	'vip_jetpack_connection_pilot_heartbeat',
	'wordpress_api_key',
	'vip_search_index_versions',
	'vip_search_global_index_versions',
];

// On multisites
"Site Meta" = [
	'vip_search_global_index_versions',
];

Custom cleanup operations

The vip_datasync_cleanup hook can be used for custom cleanup of application data (e.g., removing production API keys, performing brief data manipulation) after a data sync. On a WordPress multisite, the cleanup hook is run individually on each network site on the environment.

Custom code that uses the vip_datasync_cleanup hook should be added to a file in the /client-mu-plugins directory. Because data syncs move downward from the production environment, the vip_datasync_cleanup code must be present in the branch that deploys to the target environment (e.g. develop or staging).

In this example the vip_datasync_cleanup hook is used to delete the my_social_api_token option on the non-production environment:

/client-mu-plugins/plugin-example.php
/**
 * Run some custom cleanup after a migration.
 *
 * @uses vip_datasync_cleanup
 */
function my_action_vip_datasync_cleanup() {
    // Safety first: Don't do anything in 
    // the production environment
    if ( 'production' === VIP_GO_APP_ENVIRONMENT ) {
        return;
    }
 
    delete_option( 'my_social_api_token' );
}
add_action( 'vip_datasync_cleanup', 'my_action_vip_datasync_cleanup' );

Using the vip_datasync_cleanup hook to output logs

The data sync operation, as well as the actions defined in the vip_datasync_cleanup hook, are performed on a transient job container rather than on an environment’s app or batch containers. Because of this, if the hook is configured to output logs they will not appear in the environment’s app or batch logs.

For log output from the vip_datasync_cleanup hook to appear in an environment’s batch logs, configure the hook to create a one-time cron event with post-sync tasks. The cron event will be picked up and run on a batch container. Any logs resulting from those tasks will be output to the environment’s batch logs in Runtime Logs.

Last updated: February 28, 2024

Relevant to

  • WordPress