Skip to content

Enterprise Search

Index with Enterprise Search

Indexing is the process by which data is compiled and inserted into Elasticsearch. Elasticsearch (ES) indexes differ from MySQL table indexes. ES doesn’t store the full database, as it’s designed for quick retrieval and sorting/filtering of the items in the index.

If more than one type of ES index exists (e.g. one for terms and another one for posts), each one requires separate steps for indexing, re-indexing, and versioning.

Elasticsearch’s continual indexing is dependent on built-in WordPress event action hooks. Normally, the standard WordPress editing process uses these, and every change made by an editor to a post or page should be quickly reflected in the Elasticsearch index.

Custom code or plugins that use direct database queries or non-standard ways of modifying the database will not sync to the Elasticsearch index.

VIP-CLI command examples

For demonstration purposes, the <app-alias> value mytestsite and the <env> value develop are used in the VIP-CLI command examples below. Read more about how to target environments in VIP-CLI commands.

Content is indexed incrementally, as it is created and updated. Indexing can manually be triggered by opening the particular post or page in the editor and clicking “Update”.

The index command initiates indexing activity on a new or existing index (starting from the newest content). This is needed if an allow list entry (e.g. post types, taxonomies, or post meta) has been recently modified.

An example command and output:

$ vip @mytestsite.develop -- wp vip-search index
Indexing posts...
Processed 500/37674. Last Object ID: 37550
	...
Processed 37500/37674. Last Object ID: 179
Processed 37674/37674. Last Object ID: 1
Number of posts indexed: 37674
Total time elapsed: 605.960
Success: Done!

The amount of time to complete indexing depends on the size of the database. To increase the amount of posts indexed per cycle, use the --per-page option.

Command options

For additional reference on the options available, use the wp help vip-search index command:

vip @mytestsite.develop -- wp help vip-search index

--setup

Values not accepted.

Completely drops and recreates the index with mappings.

The --setup option is required to create an index the first time. This ensures that mappings are correctly in place.

Recreating an existing index with the --setup option or versioning is necessary to bring a database and its Elasticsearch index back into sync if:

  • The database is modified outside of normal WordPress functionality (i.e. database imports, bulk CLI commands that disable the hooks by setting WP_IMPORTING to true, or direct SQL queries that modify data).
  • Search tokenization filter settings have been added or modified.
vip @mytestsite.develop -- wp vip-search index --setup

Caution

Versioning should be used whenever possible to update a site’s index. Re-creating an existing index with the --setup option is a destructive action and can cause the index to be temporarily unavailable until the process is complete.

--network-wide

Values not accepted.

Sequentially indexes all sites on a WordPress multisite network.
Refer to Search and multisites for more information.

vip @mytestsite.develop -- wp vip-search index --network-wide

Caution

If a multisite has over 100 sites on its network, it is strongly recommend to create an index on a per-site basis. Rather than creating indexes for all sites, only create indexes for specific network sites that will actively use them. Creating unused indexes may negatively impact the indexing and revalidation operations triggered by periodic health checks.

--blog-ids

Accepted values: blog id or comma-separated list of blog ids

Will target select blog IDs.

For example, to index only blog IDs 3 and 5:

vip @mytestsite.develop -- wp vip-search index --blog-ids=3,5

--version

Accepted values: index version number, next, active, current, or previous

The index version which the command is targeting. Used during versioning to build up a new index in parallel with the current active index. Typically used in conjunction with the --indexables parameter.

For example, to index version 2 of a posts index:

vip @mytestsite.develop -- wp vip-search index --version=2 --indexables=post

--indexables

Accepted values: single value Indexable or comma-separated list of Indexables
Default values: all registered Indexables (post, user, term, comment)

Targets specific Indexable(s).

For example, to index the term Indexable:

vip @mytestsite.develop -- wp vip-search index --indexables=term

--upper-limit-object-id, --lower-limit-object-id

Accepted values: single value object ID

Limits a subset of object IDs (e.g. post IDs, term IDs, user IDs). IDs are indexed from highest to lowest.

A common use case would be to resume indexing after an interruption or timeout. For example, a previous index command was interrupted at post ID 100, so we would trigger a new index operation using --upper-limit-object-id to begin at 100:

vip @mytestsite.develop -- wp vip-search index --upper-limit-object-id=100

When these options are used together, they limit the indexing to a range of documents. For example, to index only post IDs from 50 to 10:

vip @mytestsite.develop -- wp vip-search index --upper-limit-object-id=50 --lower-limit-object-id=10

--per-page

Accepted values: numerical value of posts per page
Default value: 500, maximum of 5000

Determines the amount of posts to be indexed per cycle.

For example, to increase the indexed posts per cycle to 1000:

vip @mytestsite.develop -- wp vip-search index --per-page=1000

--include

Accepted values: single value object ID or comma-separated list of object IDs

Targets indexing only the listed ID(s).

For example, to index only 7 and 20:

vip @mytestsite.develop -- wp vip-search index --include=7,20

--post-type

Accepted values: single post type or comma-separated list of post types
Default values: all public post types

Targets indexing only the specified post type(s).

For example, to index only a custom post type event and recipe:

vip @mytestsite.develop -- wp vip-search index --post-type=event,recipe

--show-errors

Values not accepted.

Displays a verbose error output during indexing. If a post fails to index, it will return with the full error message (compared to just the post ID and title).

vip @mytestsite.develop -- wp vip-search index --show-errors

--skip-confirm

Values not accepted.

Omits the Enterprise Search confirmation prompt, “You are about to run a destructive operation. Are you sure?”, for destructive operations.

For example, indexing with --setup is a destructive action and will generate the following prompt:

$ vip @mytestsite.develop -- vip-search index --setup
⚠️  You are about to run a destructive operation. Are you sure? [y/n] y
Indexing with setup option needs to delete Elasticsearch index first, are you sure you want to delete your Elasticsearch index? [y/n] y

Including --skip-confirm will omit the destructive operation confirmation prompt:

$ vip @mytestsite.develop -- wp vip-search index --setup --skip-confirm
Indexing with setup option needs to delete Elasticsearch index first, are you sure you want to delete your Elasticsearch index? [y/n] y

Indexing progress

There are a few commands available that will allow you to check the status of the indexing or stop it:

get-indexing-status

Check on the progress of an index command that is currently running with get-indexing-status:

vip @mytestsite.develop -- wp vip-search get-indexing-status

Output will be returned in JSON format with the below fields:

  • "indexing": true for in process, false for inactive
  • "method": how the indexing was actioned
  • "items_indexed": number of items already indexed
  • "total_items": total number of items needing to be indexed
{"indexing":true,"method":"cli","items_indexed":1000,"total_items":1015153}

In a scenario where the indexing has unexpectedly failed, you may still see true for "indexing", but the total items will return -1:

{"indexing":true,"method":"cli","items_indexed":0,"total_items":-1}

get-last-indexed-post-id

To check where the last indexed post ID left off, use the get-last-indexed-post-id command:

vip @mytestsite.develop -- wp vip-search get-last-indexed-post-id

stop-indexing

An active indexing command in process can be stopped with stop-indexing:

vip @mytestsite.develop -- wp vip-search stop-indexing

delete-transient

If no indexing command is currently running (i.e. the indexing command unexpectedly stopped) and the error “Error: An index is already occurring. Try again later.” is being displayed, remove the index lock with delete-transient before re-trying to index:

vip @mytestsite.develop -- wp vip-search delete-transient

Last updated: September 08, 2022