Free tools
Robots.txt Examples
Robots.txt file content for playstation.com.
Robot.txt file for: playstation.com
# PlayStation Robots.txt User-agent: * Disallow: */explore/testing/ Disallow: */community/community-score* Disallow: */?category* Disallow: */content/usergenerated/asi/mongo/content/sites/activitystreams/activities/community/* Disallow: */search/?q* User-agent: * Crawl-delay: 5 User-agent: AhrefsBot Crawl-delay: 10 User-agent: YandexBot Crawl-delay: 10 User-agent: bingbot Crawl-delay: 10 User-agent: * Disallow: */sitemap_gpdc.xml User-agent: Algolia Crawler Allow: */sitemap_gpdc.xml User-agent: GPTBot Disallow: / User-agent: PetalBot Disallow: / # Sitemaps Sitemap: https://www.playstation.com/sitemap_index.xml