At Ahrefs, we crawl pages according to a specific algorithm that ensures an efficient use of our resources without compromising the quality of data.

So the short answer is that it can take a few days up to some weeks before we crawl your new backlink.

Some considerations of our crawl algorithm include: crawl priority (based on DR), speed of crawl or update for specific pages (based on UR), number of pages in a site and the number of new backlinks discovered.

Crawl priority for sites

In general, the more “popular” a site is (ie. a higher Domain Rating (DR), more quality backlinks), the more likely a domain will be crawled. Domains that are less “popular” (ie. lower DR, less quality backlinks) will be partially crawled.

Speed of crawl / update for specific pages

The speed of crawling and updating specific pages depends on their URL Rating (UR). Pages with higher UR have a higher chance of getting indexed and updated more often, as compared to pages with a low rating.

Number of pages in a site

Do note that if a website with a low DR has a lot of pages, we won’t be able to index all of those pages. Moreover, the re-crawling frequency will be comparatively lower.

On the other hand, if a website has a high DR score and just a few pages, our bot will be re-crawling all of them as often as possible. 

Number of new backlinks

The other important thing to note is the number of new backlinks. Once AhrefsBot discovers each new backlink pointing to a given page, it will be raising its crawling priority.

Correspondingly, if it doesn’t see any new backlinks coming to a certain page for a long time, it won’t re-crawl it as often.

Note: Our algorithm is more complex than the above-mentioned and takes into consideration other rules to determine the priority level for each particular page. 

Did this answer your question?