Site Audit allows you to notify search engines about important content changes on your website using the IndexNow protocol. This article explains how to set up and use IndexNow within Site Audit.
1. What is IndexNow?
IndexNow is a free protocol that enables website owners to inform search engines about the latest content updates, additions, or removals on their sites. By notifying search engines about these changes, you can ensure that they are aware of the updates instantly, rather than waiting for their bots to crawl and discover the changes themselves. IndexNow is currently supported by Bing, Yep, and other search engines. Learn more about IndexNow on the official website.
2. How IndexNow works
To use IndexNow, you need to prove website ownership by generating an API key and hosting a text file containing this key at the root of your website.
The API key should be between 8-128 characters long and can only contain letters (a-z, A-Z), numbers (0-9), and dashes (-). You need to create a UTF-8 text file at the root of your website with the API key as the file name and its content.
For example, if your API key is ab1405d3928e453188a66e13d632abb5, you should create a UTF-8 file at the following location: https://www.example.com/ab1405d3928e453188a66e13d632abb5.txt, and the file content should be ab1405d3928e453188a66e13d632abb5.
Once the key file is hosted, you can use this key to submit URLs to IndexNow. You can do this manually by sending HTTP requests to one of the IndexNow endpoints, as described in the official documentation. Alternatively, Site Audit can handle this process for you.
3. Setting up IndexNow in Site Audit
You can set up IndexNow in Site Audit by navigating to Project settings > Site Audit > Crawl settings, and then locating the IndexNow section.
If you don't have an API key, you can generate a new one directly in this section. However, don't forget to host the key file at the root of your website, as mentioned above. If you already have a key, simply paste it into the input box.
Please ensure that your website is accessible to AhrefsBot, so we can crawl and verify the key file.
After saving the settings, you can submit URLs to IndexNow from your Site Audit report.
4. Submitting URLs to IndexNow
To submit multiple URLs in bulk, open Page Explorer, filter the URLs you want to submit, and click the "IndexNow" button. This will submit all the URLs in the current report to IndexNow.
To avoid wasting your crawl budget, we recommend ensuring that all submitted pages have relevant and positive changes. You can review the pages recommended for submission in the pre-configured "Pages to submit to IndexNow" issue. This issue automatically selects pages with content changes, as well as newly redirected or removed pages. You can also create your own custom issue with specific filter conditions or exclude certain pages from the report.
To submit an individual URL to IndexNow, open the URL details panel and click the "IndexNow" button.
After submitting the URLs, you'll receive a success notification. If there's an issue, such as a missing key file on your site, Site Audit will inform you about the error.
By following these steps, you can efficiently notify search engines about important content changes on your website using the IndexNow protocol within Site Audit.