Sometimes, our bot might be blocked by the Cloudflare Firewall preventing it from crawling the website.
If you want to lift this restriction, follow the steps below.
- Log in to Cloudflare admin;
- Open Firewall app;
- Enter our IP ranges separated by comma, change Block to Whitelist and click the Add button:
Here's the detaled article from Cloudflare support:
https://support.cloudflare.com/hc/en-us/articles/217074967-How-do-I-control-IP-access-to-my-site-
Related articles: