Our Crawler
Some of our vehicle fitment data is collected by an automated crawler. We run it conservatively and transparently.
How it behaves
- It identifies itself with a descriptive user-agent.
- It respects
robots.txtand crawl-delay directives. - It rate-limits requests and caches responses to avoid load on source sites.
- It only collects factual specification data, with the source recorded for each record.
Are you a site owner?
If you'd prefer we don't crawl your site, or you've spotted our crawler behaving badly, email bot@comparewheels.com and we'll address it promptly.