Sometimes, our bot might be blocked by the Cloudflare Firewall preventing it from crawling the website.
If you want to lift this restriction, follow the steps below.
Log in to Cloudflare admin;
Open Firewall app;
Enter our IP ranges separated by comma, change Block to Whitelist and click the Add button:
Here's the detailed article from Cloudflare support:
https://support.cloudflare.com/hc/en-us/articles/217074967-How-do-I-control-IP-access-to-my-site-
Related articles: