User Agent Restrictions

Access to individual environments of an application can be restricted by specifying a list of user agents to deny access in the VIP Dashboard’s User Agent Restrictions panel. These settings are useful for sites with highly sensitive content, intranets, and non-production environments in cases where a tool or bot sends suspicious requests from various IP addresses while retaining the same User Agent.

Requests can be blocked based on exact or partial matches of user agent strings. Block rules are case insensitive. For example, the rule webscraper will match all of the following user agents: WEBSCRAPER, WEBscraper, and webscraper.

When blocking by User Agents, take care not to block common browser User Agents, or User Agents that look similar. It is a best practice to search recent HTTP Request Logs to validate that full, or partial matching blocking will only impact the intended requests. Blocking requests from common browser User Agents can result in blocking real user requests.

Limitations

• Depending on the particular configuration of the User Agent Restriction List, external services and tools that connect to a particular application may be blocked. If this happens, an adjustment must be made to the active restriction list to accommodate for access needs.
• Be careful not to block user agents associated with VIP services which require access to the application.
• An User Agent Control List may have no more than 250 entries. If you require additional entries, please open a ticket with our Support Team.

Types of requests restricted by the User Agent Restriction List

Once enabled, the User Agent Restriction List will reject any requests with user-agent HTTP headers that match the supplied restriction rules with a 403 Forbidden error response from VIP’s CDN. This includes requests of the following types:

• both cached and uncached requests
• requests from logged in and anonymous users
• for static files, media files, and dynamically generated content, including REST API endpoints
• for a WordPress or a Node.js application

The only exception to the list of restricted requests above, is requests from services within Automattic’s networks. These requests require access to support the operation of a VIP application.

Edit the User Agent Restriction List

The User Agent Restrictions panel is located in the application view of the VIP Dashboard. User agent restrictions are managed separately for each environment of an application. By default, all environments will have an empty User Agent Restriction list.

Changes made to the User Agent Restriction List will take 5-10 minutes to apply to the environment.

1. Navigate to the VIP Dashboard.
2. Select an environment from the environment dropdown located at the top of the VIP Dashboard navigation menu to which the settings will apply.
3. Select “Security Controls” from the sidebar navigation at the left of the screen.
4. Select “User Agent” from the submenu.
5. The User Agent Restrictions panel will display any existing settings. If no restrictions are currently added, the User Agent Restriction List will be blank.
6. To add entries to the restriction list, select the “+ Add” button. This will bring up a dialog that requests a list of user agent identifier strings to restrict along with an optional note for additional context. These identifier strings will be matched exactly if equals is selected or partially if contains is selected.
7. To edit your existing list, select “Edit full list” above the restriction list. This will bring up a dialog showing the full list of restrictions in the format of [contains|equals] [user agent identifier string] #[Optional Note]. After making necessary edits to this list, select Save to accept these changes or Cancel to reject them.

Restriction Rule Syntax

The required format for editing the full list of user agent restrictions is [contains|equals] [user agent identifier string] #[Optional Note].

• [contains|equals] – Choose contains to restrict access to any request that contains the supplied user agent identifier string. Choose equals to restrict access to any request that matches the supplied user agent identifier string exactly.
  
  For example, to block requests with the user-agent HTTP header of Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.1; +https://openai.com/gptbot , the full user agent string would need to be added to an equals entry. However, blocking user agents that contain the user agent string of openai would also block a request from this user agent, as well as any other requests that contain openai.
• [user agent identifier string] – The full or partial user-agent HTTP header to be restricted from accessing the application.
• [Optional Note] – Adding a note allows additional context to be added to each restriction rule. Rules with the same optional rule will be grouped together.

Edit or Remove Individual User Agent Restriction entries

It is possible to edit and remove individual user agent control entries without editing the full list of restrictions.

1. Navigate to the VIP Dashboard.
2. Select an environment from the environment dropdown located at the upper left of the VIP Dashboard to which the settings will apply.
3. Select “Security Controls” from the sidebar navigation at the left of the screen.
4. Select “User Agent” from the submenu.
5. Select “Edit” located to the right of the existing entry to edit this entry or select the trashcan icon to delete this particular entry.
6. Select “Save“.

To completely disable the User Agent Restrictions feature and allow the environment to be accessible from anywhere on the internet, all User Agent restriction entries must be removed.