Skip to main content

Block Unwanted Requests Examples

With Traffic Policy, you can block unwanted requests to your endpoints. This page demonstrates a few example rules that do so.

See the following Traffic Policy action docs for more information:

How do I deny traffic from Tor?​

This rule uses the connection variables available in IP Intelligence to block Tor exit node IPs.

Loading…

How do I deny traffic from bots and crawlers with a robots.txt?​

This rule returns a custom response with a robots.txt file to deny search engine or AI crawlers on all paths.

Loading…

You can extend this example to create specific rules for crawlers based on their user agent strings, like ChatGPT-User and GPTBot.

Loading…

How do I block traffic from bots and crawlers by user agent?​

You can also take action on incoming requests that contain specific strings in the req.user_agent request variable.

Loading…
tip

You can expand the expression to include additional user agents by extending the (chatgpt-user|gptbot) section of the regular expression.

Loading…

How do I block traffic from bots and crawlers by IP Address?​

You can also use IP Intelligence variables to block AI Bots by IP Address.

Loading…

Deny non-GET requests​

This rule denies all inbound traffic that is not a GET request.

Loading…

Custom response for unauthorized requests​

This rule sends a custom response with status code 401 and body Unauthorized for requests without an Authorization header.

Loading…

How do I block traffic from specific countries?​

Sometimes you may need to block requests originating from one or more countries to remain compliant with data regulations or sanctions. This rule blocks requests based on the origin country using ISO country codes with the following steps:

  1. Check if the request is from an array of countries you can define
  2. If so, return a 401 status code with an error message.
Loading…

Limit request sizes​

This rule demonstrates how to prevent excessively large user uploads, like text or images, that might cause performance or availability issues for your upstream service with the following steps:

  1. Check if the request is POST or `PUT
  2. Check if the request's content is 1MB or larger.
  3. If both conditions are met, return a 400 status code with an error message.
Loading…

Exempt specific traffic from rate limits​

In this example, the Algolia web crawler is exempted from any rate limiting we have on our site. See the IP Intelligence docs for other bots and crawlers that are available.

Loading…