Data sync from production to non-production environments

Data syncs between VIP Platform environments facilitate testing and quality assurance (QA) of new features and allows teams to accurately reproduce and examine errors in a non-production environment. After a data sync from a production environment to a non-production environment is completed, the target environment will automatically load shared media files that were uploaded to production. This eliminates the need to copy media between environments.

Limitations

• Data syncs can only occur from a production environment to a non-production environment.
• Data syncs from the most recent backup copy of a production database, not from the primary database.
• WordPress multisite environments that have sites created in addition to the main site (ID 1) must create and maintain a config.yml file for the results of a data sync to work as expected.
• After a data sync has completed, non-production environments have access to a production environment’s shared media files in read-only mode.
• Elasticsearch indexes are not included in the data sync process.  

Actions that occur during a data sync

The data sync operation is designed not to impact the production environment in any way. The following actions only occur on the target environment:

1. The target environment is protected via “maintenance mode”. This prevents access to the environment while the data is being restored and manipulated, and ensures that no changes can be made (and lost) during this time.
2. The most recent database backup of the production environment is written to the database of the target environment. This process overwrites existing tables, but does not remove tables that are unique to the target environment.
3. Occurrences of the production environment’s convenience domain or primary domain in the database are replaced with the primary domain of the target environment. If a data sync config file exists for the target environment, additional search and replace operations will occur as per the config file.
   • For example, if the production environment’s primary domain is  example.com, and the target environment’s primary domain is develop.example.com, then all instances of example.com will be replaced with develop.example.com in the target database after the sync.
   • For target environments that do not yet have a primary domain assigned,  example.com will be replaced by the target environment’s convenience domain (e.g. example-com-develop.go-vip.net).
4. Protected options are restored.
5. A wp vip data-cleanup datasync CLI command is run for further cleanup.
6. The vip_datasync_cleanup hook can be used to perform custom cleanup of application data at this stage.
7. Jetpack is (re)connected.
8. The object cache is flushed.
9. Maintenance mode is deactivated.

Initiate a data sync

vip @example-app.develop sync

Protected options

A specific set of options that exist on the target environment are backed up before the data sync begins, then restored after the data sync has completed. This prevents environment-specific configurations from being shared between environments.

The protected options list cannot be extended or modified.

"WP Options" = [
	'jetpack_options',
	'jetpack_private_options',
	'vip_jetpack_connection_pilot_heartbeat',
	'wordpress_api_key',
	'vip_search_index_versions',
	'vip_search_global_index_versions',
	'parsely',
];

// On multisites
"Site Meta" = [
	'vip_search_global_index_versions',
];