Use Enterprise Search on a VIP Local Development Environment

Set up a local environment

Follow the steps to create a VIP Local Development Environment as

• a multisite
• with the custom slug value example-site
• and the --app-code option set to the path of a git clone of an application’s GitHub repository on the local machine.

In this command example, the location of a git clone of vip-go-skeleton on the local machine is assigned to --app-code:

vip dev-env create --multisite=y --app-code="/Users/user/vip-go-skeleton" --slug=example-site

1. After the environment is created and started, log in to the backend with the generated URL:
   http://example-site.vipdev.lndo.site/wp-admin
   The local environment is set up as an empty multisite (with one main site).
2. Navigate to the home page: http://example-site.vipdev.lndo.site
   After logging in, the WordPress admin tool bar will be visible at the top of the browser window.

Enable Enterprise Search

1. Navigate to the vip-config directory.

cd vip-go-skeleton/vip-config

2. To enable both Enterprise Search and Search Dev Tools, add the code below to vip-config.php:

/**
 * Enable VIP Search
 *
 */
define( 'VIP_ENABLE_VIP_SEARCH', true );
define( 'VIP_ENABLE_VIP_SEARCH_QUERY_INTEGRATION', true );

Test Enterprise Search

After completing the above steps, Elasticsearch is now enabled for the site. Perform a basic test to verify that Enterprise Search is running as expected.

1. Reload the page with the search results: http://example-site.vipdev.lndo.site/?s=example
2. Locate the Search Dev Tools dropdown link in the admin tool bar. It will appear to the right of Query Monitor dropdown link

3. Click the Search Dev Tools and expand the query:

On the VIP Local Development Environment, Enterprise Search automatically starts in the background (not enabled yet) and existing posts at that time were indexed.

Test indexing

1. To verify if indexing is working, publish a new test post with title of “New test post” with content “This post contains some words to search for. Who likes cheeseburgers? What about hot dogs?”.

2. Perform a search again with text that was in the new test post: http://example-site.vipdev.lndo.site/?s=cheeseburgers

3. Use the CLI to check the health of the index with the validate-counts command.

vip dev-env exec --slug=example-site -- wp vip-search health validate-counts

The output from the command will display post counts for two different post types, post and page, and compare the count for each between the database (DB) and index (ES).

Validating post count

✅ no inconsistencies found when counting entity: post, type: post, index_version: 1 - (DB: 2, ES: 2, Diff: 0)
✅ no inconsistencies found when counting entity: post, type: page, index_version: 1 - (DB: 1, ES: 1, Diff: 0)

The health check above shows that there are no differences between the database and the index. This indicates that Enterprise Search is enabled and indexing is running correctly.

Add an Enterprise Search index for a new network site

1. Create a new site on the network by navigating to “Sites” in the Network Admin:
   http://example-site.vipdev.lndo.site/wp-admin/network/sites.php
   and selecting “Add New”.
2. Complete the field prompts with the following values:
   • Site Address (URL): new-example-site
   • Site Title: New Example Site
   • Admin Email: example.admin@example-site.com (or a preferred email value)

3. Select “Add Site“.

4. Navigate to the home page of the site on the front end.
5. Enter the search term “hello” in the Search field.
   http://new-example-site.example-site.vipdev.lndo.site/?s=hello
6. Open the Search Dev Tools panel to view more information about the search results.
   Search Dev Tools shows that an index exists for the new site, but no posts have been indexed.

7. Use the CLI to check the health of the site index with the validate-counts command.

vip dev-env exec --slug=example-site -- wp vip-search health validate-counts --url=new-example-site.example-site.vipdev.lndo.site

The health report indicates there are no posts in the site index:

Validating post count

🟧 skipping, because there are no documents in ES when counting entity: post, type: post, index_version: 1
🟧 skipping, because there are no documents in ES when counting entity: post, type: page, index_version: 1

8. Run the index command with the --setup option to sync the posts in the database with the index.

Note: When re-indexing a live production site, use versioning. Since this is a local environment with no live traffic, we will index as-is.

vip dev-env exec --slug=example-site -- wp vip-search index --setup --url=new-example-site.example-site.vipdev.lndo.site

⚠️  You are about to run a destructive operation. Are you sure? [y/n] y
Adding post mapping…
Success: Mapping sent
Indexing posts...
Processed 2/2. Last Object ID: 1
Number of posts indexed: 2
Total time elapsed: 0.524
Success: Done!

9. Search for “hello” again in the site: http://new-example-site.example-site.vipdev.lndo.site/?s=hello

The search result will now be displayed.

10. The Search Dev Tools will also return a search result:

Offload to WP_Query

Non-search queries are not offloaded to Enterprise Search by default. In order to offload them to Enterprise Search, WP_Query will need to be modified.

1. Generate some posts with content using the command line.

curl -N http://loripsum.net/api/5 | vip --slug=example-site dev-env exec -- wp post generate --post_content --count=10

This generates 10 new posts with Lorem Ipsum content in the main site (site ID 1). The posts will be indexed immediately.

2. Repeat the same command with an older date using the --post-date option.

curl -N http://loripsum.net/api/5 | vip --slug=example-site dev-env exec -- wp post generate --count=10 --post_type=post --post_date=1999-01-04

3. Navigate to the home page to view the newly generated posts: http://example-site.vipdev.lndo.site

4. Navigate to the second page of posts: http://example-site.vipdev.lndo.site/page/2/
5. Open the Query Monitor panel and view “Queries”. Look for the query that determines which posts are displayed on the second page:

SELECT SQL_CALC_FOUND_ROWS wp_posts.ID
 FROM wp_posts
 WHERE 1=1
 AND wp_posts.post_type IN ('post', 'page')
 AND (wp_posts.post_status = 'publish'
 OR wp_posts.post_status = 'private')
 ORDER BY wp_posts.post_date DESC
 LIMIT 10, 10

If the query does not appear, the object cache may need to be flushed with wp cache flush.

6. In the footer, the “Recent Posts” widget displays the latest five posts. The query for this widget can also be found in Query Monitor:

SELECT wp_posts.ID
 FROM wp_posts
 WHERE 1=1
 AND wp_posts.post_type = 'post'
 AND ((wp_posts.post_status = 'publish'))
 ORDER BY wp_posts.post_date DESC
 LIMIT 0,5

The query for the “Recent Posts” widget and for the posts on the second page are usually relatively fast, but they can become slow on a site with a lot of posts and traffic.

7. On the local machine, navigate to the client-mu-plugins directory in the vip-go-skeleton directory and create a new file named search.php.

cd /vip-go-skeleton/client-mu-plugins
touch search.php

8. Add the below code to search.php. The code in this snippet uses the pre_get_posts hook to offload both queries (e.g. not of the back-end or category type):

<?php

/**
 * Offload to ES
 *
 */
function vip_sample_es_offload( $query ) {
	if ( is_admin() ) {
		return;
	}
	
    if ( ! $query->is_category() ) {
        $query->set( 'es', true );
    }
}
add_action( 'pre_get_posts', 'vip_sample_es_offload' );

9. Navigate to the homepage and select the Search Dev Tools panel from the WordPress admin tool bar: http://example-site.vipdev.lndo.site
   

10. Refer to “Queries” in the Query Monitor panel. Locate the /* ES_WP_Query Shoehorn */ comment at the end of the query. This message indicates that the query was generated by es-wp-query:

When Elasticsearch returns the list of post IDs that matched the query, the database is queried (via the “shoehorn”) to obtain the post data needed to construct the HTML result that a user sees on the front end.

Elasticsearch queries run first, and any other MySQL queries (without the shoehorn comment) run afterward.

Remove the environment

When no longer needed, the local environment can be completely removed from the local machine with the command:

vip dev-env destroy --slug=example-site