Free tools
Robots.txt Examples
Robots.txt file content for bandcamp.com.
Robot.txt file for: bandcamp.com
User-agent: * Disallow: /tools Disallow: /checkout Disallow: /download_check Disallow: /cart/ Disallow: /corpbanner/ Disallow: /stream Disallow: /api/ Disallow: /design_tokens Disallow: /search Allow: /api/currency_data/ Allow: /api/discover/1/discover_mobile_web Allow: /api/discover/1/discover_web Allow: /api/tag_search/2/related_tags Disallow: /*_cb$ User-agent: NextGenSearchBot Disallow: / User-agent: EdisterBot Disallow: / User-agent: Ezooms Disallow: / User-agent: SWEBot Disallow: / User-agent: discobot Disallow: / User-agent: SemrushBot Disallow: / User-agent: grapeshot Disallow: / User-agent: BUbiNG Disallow: / User-agent: Bytespider Disallow: / User-agent: CCBot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Diffbot Disallow: / User-agent: FacebookBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: GPTBot Disallow: / User-agent: omgili Disallow: /