Once you've run a crawl of your site in the Site Audit tool, you might see a '5XX page' error in the report under the Internal pages menu:
What triggers this issue?
This issue reports all internal page URLs of a website returning one of the 5xx (Server Error) HTTP response codes. Click here for the full list of 5XX Server errors.
Why is it important?
URLs returning one of the 5xx status codes can be accessed neither by your website visitors not by the search engines crawlers. Crawlers will be forced to abandon the request while people will most likely leave your website.
This can damage your website crawlability (and thus indexability) and will negatively affect the user experience on your website.
How to fix it
Review the list of 5xx URLs.
Try to reproduce the server error reported by the Site Audit for these URLs in your browser to see if the problem still resists. You should also check the error logs for your server.
One of the most common 5xx errors is 503 (Service Unavailable) when the web server was unable to handle the request due to a temporary overloading or maintenance. Such temporal issues do not require any extra work and, as a rule, get resolved with the next successful crawl.
Your server may be overloaded or misconfigured. If this is an ongoing problem, you might need to check with your hosting provider or with your web developers.
One of the possible reasons for this issue in Site Audit tool is a high crawling speed set in your project's settings. Some servers are not able to process a huge number of requests at a given time. In this case, to fix it you need to try to set a lower crawling speed and run a project re-crawl.
Also, some part of your website can be blocked from our crawler (because of a firewall, DoS protection system, traffic blocks from certain countries, or a CMS configuration). Thus your website URLs may return 5xx codes to our crawler specifically.
- What is the optimal crawl speed I should set in Site Audit?
- What does '500 page' error in Site Audit mean?
- Canonical points to 5XX