Avoid post__not_in

The WP_Query argument post__not_in appears to be a helpful option, but it can lead to poor performance on a busy and/or large site due to affecting the cache hit rate.

It’s usually used to exclude specific post IDs from a query’s results. For instance, if you’ve got a widget that shows the most recent 5 posts on every page, your site designer may want to avoid showing the current post in that widget – it’s a bit redundant, as the reader is already reading that recent post.

How it’s used

You might have a widget like this:

// Display the most recent news posts
function my_recent_news_widget( ) {
	$args = array(
		'category_name' => 'news',
		'posts_per_page' => 5,
		'post_status' => 'publish',
		'ignore_sticky_posts' => true,
		'no_found_rows' => true,
	);
	$recent_posts = new WP_Query( $args );

	echo '<div class="most-recent-news"><h1>News</h1>';
	while ( $recent_posts->have_posts() ) {
		$recent_posts->the_post();
		the_title( '<h2><a href="' . get_permalink() . '">', '</a></h2>');
	}
	echo '</div>';
	wp_reset_postdata();
}

The typical approach is to modify that function alone, adding an optional function argument and a post__not_in query argument on line 9, as follows:

// Display the most recent news posts (but not the current one)
function my_recent_news_widget( $exclude = array() ) {
	$args = array(
		'category_name' => 'news',
		'posts_per_page' => 5,
		'post_status' => 'publish',
		'ignore_sticky_posts' => true,
		'no_found_rows' => true,
		'post__not_in' => $exclude,
	);
	$recent_posts = new WP_Query( $args );

	echo '<div class="most-recent-news"><h1>News</h1>';
	while ( $recent_posts->have_posts() ) {
		$recent_posts->the_post();
		the_title( '<h2><a href="' . get_permalink() . '">', '</a></h2>');
	}
	echo '</div>';
	wp_reset_postdata();
}

You’d probably call this in a template with my_recent_news_widget( [ get_the_ID() ] ); and it would display the most recent 5 news posts, but not the current post.

But while simple, this is not good!

Problems with this approach

The query, which was previously leveraging the built-in query cache, is now unique for every post or page due to the added AND ID not in ( '12345' ). This is due to the cache key (which is a hash of the arguments) now including a list of at least one ID, and that is different across all posts. So instead of subsequent pages obtaining the list of 5 posts from the object cache, it will miss the cache, and the database will do the same work on multiple pages.

As a result, each of those queries is now cached separately, unnecessarily increasing Memcached use. For a site with hundreds of thousands of posts, this will potentially impact the object cache size and result in premature cache evictions.

What to do instead

In almost all cases, you can gain significant speed improvements by requesting more posts and skipping the excluded posts in PHP.

You can improve performance by ensuring the post query is consistent across all the URLs, retrieving the most recent 6 posts, so that it’s retrieved from the object cache. If you anticipate the $exclude list to be larger than one post, set the limit higher, perhaps to 10. Make it a fixed number, not a variable, to reduce the number of cache variants.

The updated function no longer excludes the post(s) in SQL, it uses conditionals in the loop:

// Display the most recent news posts (but not the current one)
function my_recent_news_widget( $exclude = array() ) {
	$args = array(
		'category_name' => 'news',
		'posts_per_page' => 10,
		'post_status' => 'publish',
		'ignore_sticky_posts' => true,
		'no_found_rows' => true,
	);
	$recent_posts = new WP_Query( $args );

	echo '<div class="most-recent-news"><h1>News</h1>';
	$posts = 0; // count the posts displayed, up to 5
	while ( $recent_posts->have_posts() && $posts < 5 ) {
		$recent_posts->the_post();
		$current = get_the_ID();
		if ( ! in_array( $current, $exclude ) ) {
			$posts++;
			the_title( '<h2><a href="' . get_permalink() . '">', '</a></h2>');
		}
	}
	echo '</div>';
	wp_reset_postdata();
}

While requiring a bit of logic in PHP, this approach leverages the query cache better and avoids creating many cache variations that might impact the site’s scalability and stability.