PeptideCompassBot

Reads publicly published, unauthenticated product and catalog pages on peptide vendor storefronts — the same pages any visitor's browser can load.

A per-vendor price/catalog crawl runs weekly. A separate monthly pass refreshes vendor profile facts (news, enforcement actions, published reviews). The weekly price crawl itself calls no LLM at steady state.

Extracts factual fields only — price, pack size, stock status, stated purity. It does not republish a vendor's marketing copy or images.

• Never creates an account, logs in, or accesses member/wholesale pricing behind a login.
• Never submits a form, checks out, or otherwise interacts with a page beyond reading it.
• Never solves a CAPTCHA or defeats a real access-control or paywall.
• Never collects personal data about a vendor's site visitors or staff.
• Never scrapes a vendor that has opted out or is under legal hold.
• Never feeds a vendor's paid placement — no crawl output can be purchased into rank or position.

User-Agent

PeptideCompassBot/1.0 (+https://peptidecompass.net/bot)

Concurrent requests per domain

1

Minimum delay between requests

2,000 ms (≈0.5 req/s)

Scope

Public, unauthenticated pages only

One request in flight per domain, at least two seconds apart, honestly identified. A vendor's own `Crawl-delay` is honored where it asks for something slower. This is the same politeness standard a well-behaved search-engine indexer follows.

If you run a vendor site and would rather PeptideCompassBot not crawl it, email crawler-optout@peptidecompass.net with the domain. We add it to our blocklist and the next scheduled run skips it — no further crawling, logged and reversible if you change your mind later.