Our Crawler

Some of our vehicle fitment data is collected by an automated crawler. We run it conservatively and transparently.

How it behaves

• It identifies itself with a descriptive user-agent.
• It respects robots.txt and crawl-delay directives.
• It rate-limits requests and caches responses to avoid load on source sites.
• It only collects factual specification data, with the source recorded for each record.

Are you a site owner?

If you'd prefer we don't crawl your site, or you've spotted our crawler behaving badly, email bot@comparewheels.com and we'll address it promptly.