At Ahrefs, we crawl the entire web 24/7, storing petabytes of information about live and broken pages.

In Content Explorer, when our crawler downloads a page, data is sent to our index at the time of the crawl. These values are not updated until we crawl the page again. We call this cached data.

In Site Explorer, on the other hand, we use a different index where values are updated regularly. We call this live data.

When you filter or sort a report in Content Explorer, cached data is used to load your results. After the results are loaded, we then update these values with their live counterparts from Site Explorer's index. This is why you can see unexpected results, and the order shown may look random.


Note: Results shown when using the "Only broken" filter are much more prone to discrepancies between cached and live values. If you hover over the "404" label, you'll see the date on which our crawler last found the page to be live. The earlier that date, the higher the discrepancy you will likely see.

To see the original values stored in Content Explorer, you can export the report.

Did this answer your question?