Skip to main content

How to whitelist Ahrefs IPs in Cloudflare?

Find out how to whitelist Ahrefs IP in Cloudflare Firewall

Written by Anna

Sometimes, our bot: AhrefsBot might be blocked by the Cloudflare Firewall preventing it from crawling the website. 

If you want to lift this restriction, follow the steps below.


Whitelist using a custom rule

  1. Log in to Cloudflare and select the domain.

  2. Go to Security > Security rules.

  3. Click Create rule, then choose Custom rules.

  4. Give the rule a name, for example “Ahrefs IPs.”

  5. Under When incoming requests match... section, set Field to “IP Source Address,” Operator to “is in,”:

    1. Refer to the list of ranges from https://help.ahrefs.com/en/articles/78658-what-is-the-list-of-your-ip-ranges "Our crawlers use the following ranges:" section

    2. Return to Cloudflare page and press Edit expression on the right side of Expression preview

    3. Paste the list of ranges between curled brackets {}

    4. Copy the list of individual IP addresses from https://help.ahrefs.com/en/articles/78658-what-is-the-list-of-your-ip-ranges "And the following individual IP addresses outside these ranges:"

    5. Go back to Cloudflare page, put cursor after 202.8.40.0/22 and before }) , press Space bar

    6. Paste the list of individual addresses from clipboard

    7. Press Use expression builder to see the list of networks and IP addresses in a nicer view.

  6. Under Then take action section, set the action to “Skip.”

  7. Under WAF components to skip section, enable: "All remaining custom rules, All rate limiting rules, All managed rules, and All Super Bot Fight Mode rules".

  8. Optionally keep “Log matching requests” enabled.

  9. Set placement order as needed and deploy the rule.

You can refer to this image to confirm the rule's setup:

This should work for most websites with a generic setup. If it does not work for you, please contact Cloudflare Support.
​

If you have Cloudflare with Bot management

  1. In the When incoming requests match... section:

    1. select Field = "Bot detection", Operator = "is in", Value = (any(cf.bot_management.detection_ids[*] in {121451232 133933265}) and cf.bot_management.verified_bot)

    2. Add Field = "Verified Bot", and toggle it on.

  2. In the Then take action section:

    1. Select Choose action = "Skip"

  3. Under Then take action section, set the action to “Skip.”

  4. Under WAF components to skip section, enable: "All remaining custom rules, All rate limiting rules, All managed rules, and All Super Bot Fight Mode rules".

  5. Optionally keep “Log matching requests” enabled.

  6. Set placement order as needed and deploy the rule.


​

Related Articles
Did this answer your question?