About PricebosBot

PricebosBot is the crawler run by Pricebos. It only visits websites of stores that have signed up with Pricebos and asked for their prices to be read from their product pages.

How it works

• It follows robots.txt, including Crawl-delay.
• It opens at most one page every 2 seconds per website.
• It only reads sitemaps and product pages: name, price, stock and product codes.
• It pauses when a server answers HTTP 429 or 503.

Stopping the bot

Add these lines to your website's robots.txt:

User-agent: PricebosBot
Disallow: /

Or contact us through the Contact page.