When you validate a site using Rocket Validator, your server will be visited by the following systems:
- Rocket Validator Spider
- It will start at the initial URL to discover the internal links, and visit all of them until it finds all the pages to validate.
- Rocket Validator HTML checker
- It will validate HTML / CSS markup on each of your pages.
- Rocket Validator Accessibility checker
- It will validate Accessibility on each of your pages (optional, Pro feature).
This means that, for each page on your site, your server is going to at least 2 visits (scraper and HTML checker), maybe 3 if you’re also checking Accessibility.
In case your server is protected by a rate limiter or a firewall, this can result in Rocket Validator being blocked. The way to ensure RV doesn’t get blocked on your server in this case is by explicitly allowing RV to visit your site.
As our IPs are dynamic, the proper way to do this is by identifying our requests by the User Agent header we include in our requests, which currently are the following:
- Rocket Validator Spider User Agent
- Rocket Validator (+https://rocketvalidator.com)
- Rocket Validator HTML checker User Agent
- Rocket Validator Accessibility checker User Agent
- Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/68.0.3419.0 Safari/537.36