Retrieving remote data

Fetching data from other servers (e.g., fetching information from external APIs or resources) can be a relatively slow process, and issues relating to timeouts can occur. Remote calls such as wp_remote_get(), wp_safe_remote_get() and wp_oembed_get() should rely on the WordPress HTTP API (not cURL) and should be cached.

By using the HTTP API, several different transport methods (curl and PHP streams by default) can be used in code as well as a wide variety of actions and filters to later modify the request.

wpcom_vip_file_get_contents()

wpcom_vip_file_get_contents() works much like PHP’s built-in file_get_contents() function and returns either the HTML result as a string or false on failure. However, it caches and even returns previously cached data if a new remote request fails. Using this function is strongly recommended for any remote request that does not require receiving fresh, up-to-the-second results (i.e. anything on the front end of a site).

Arguments that can be passed:

• $url (string): Required. The URL from which to fetch data.
• $timeout (int): Optional. The timeout limit in seconds. Accepts values between 1 and 10; defaults to 3. Because remote requests must complete before the rest of the page will load, a timeout greater than 3 seconds is strongly discouraged.
• $cache_time (int): Optional. The minimum cache time in seconds. Accepts values of 60 or higher; defaults to 900 (15 minutes). Setting this higher will result in a faster site as remote requests are relatively slow. Results may be cached even longer if the remote server sends a cache-control header along with its response, and if the cache-control value is larger than the value assigned to $cache_time.
• $extra_args (array): Optional. Can be used to set an array of advanced arguments for additional configuration options.

Like PHP’s file_get_contents() function, wpcom_vip_file_get_contents() returns the result. To output the result, use echo.

The following code snippet is a demonstration of example values set for all four argument options for wpcom_vip_file_get_contents():

echo wpcom_vip_file_get_contents(
	'http://example.com/file.txt',
	3,
	900,
	array( 'obey_cache_control_header' => false )
);

obey_cache_control_header()

By default, if the remote server sends a cache-control header with a max-age value that is larger than the cache time passed as the third parameter of the obey_cache_control_header() function, then this remotely provided value will be used instead. This is because it’s assumed that it’s safe to cache data for a longer period of time if the remote server says the data is not going to change. If you wish to ignore the remote server’s header response and forcibly cache for only the time specified by the third parameter, then a function call along these lines should be used:

http_api_args

http_api_args can pass several different arguments (e.g., custom headers, cookies) directly to the wp_remote_get() call. Example usage:

echo wpcom_vip_file_get_contents(
	'http://example.com/file.txt',
	3,
	900,
	array(
		'http_api_args' => array(
			'headers' => array(
				'Accept-Encoding' => 'gzip',
			) 
		) 
	) 
);

vip_safe_wp_remote_get()

vip_safe_wp_remote_get() is a sophisticated extended version of wp_remote_get(). It is designed to more gracefully handle failure than wp_safe_remote_get() does.

Arguments that can be passed:

• $url (string): Required. The URL from which to fetch data.
• $fallback_value (string): Optional. To set any of the next arguments, pass an empty string, '' for this argument.
• $threshold (int): Optional. The number of fails required before subsequent requests automatically return the fallback value. This prevents continually making requests and receiving timeouts for a down or slow remote site. Accepts values between 1 and 10; defaults to 3.
• $timeout (int): Optional. The number of seconds before the request times out. Accepts values between 1 and 3; defaults to 1.
• $retry (int): Optional. This argument controls both the number of seconds before resetting the fail counter and the number of seconds to delay making new requests after the fail threshold is reached. Accepts values of 10 or higher; defaults to 20.
• $args (array): Optional. Can pass several different arguments to wp_remote_request().

Example usage of vip_safe_wp_remote_get():

// Get a URL with a 1 second timeout and cancel remote calls for
// 20 seconds after 3 failed attempts in 20 seconds have occurred
$response = vip_safe_wp_remote_get( $url );
if ( is_wp_error( $response ) ) {
	echo 'No data is available.';
} else {
	echo wp_remote_retrieve_body( $response );
}

// Get a URL with 1 second timeout and cancel remote calls for 60 seconds
// after 1 failed attempt in 60 seconds has occurred. On failure, display 'N/A'.
$response = vip_safe_wp_remote_get( $url, false, 1, 1, 60 );
if ( is_wp_error( $response ) ) {
	echo 'N/A';
} else {
	echo wp_remote_retrieve_body( $response );
}

fetch_feed()

To fetch and parse feeds, the fetch_feed() function built into WordPress should be used. fetch_feed() has built-in caching that defaults to 43200 seconds (12 hours). This value can be modified programmatically with a custom filter. For example:

function someprefix_return_900() {
	return 900;
}

add_filter( 'wp_feed_cache_transient_lifetime', 'someprefix_return_900' );
$feed = fetch_feed( $feed_url );
remove_filter( 'wp_feed_cache_transient_lifetime', 'someprefix_return_900' );

wpcom_vip_wp_oembed_get()

wpcom_vip_wp_oembed_get() is a wrapper for the WordPress function wp_oembed_get() but with added caching.

Uncached remote requests

If for some reason an uncached remote request is required (e.g., to ping an external service during post publish) then the WordPress HTTP API should be used rather than directly using cURL or another method.