Free tools
Robots.txt Examples
Robots.txt file content for coursera.org.
Robot.txt file for: coursera.org
User-agent: * Allow: /api/utilities/v1/imageproxy Disallow: /maestro/api/ Disallow: /api/ Disallow: /maestro/ Disallow: /ui/ Disallow: /signature/voucher/ Disallow: /account/ Disallow: /acclaimbadge/ Disallow: /voucher/ Disallow: /search Disallow: /ent-website/ Disallow: /learn-perf/ Disallow: /specializations-perf/ Disallow: /professional-certificates-perf/ Disallow: /learn-noperf/ Disallow: /specializations-noperf/ Disallow: /professional-certificates-noperf/ Disallow: /career-academy/programs/ Disallow: /business/xmlrpc.php Disallow: /business/wp-content/uploads/ Sitemap: https://www.coursera.org/sitemap.xml User-agent: CCBot Disallow: /lecture/ User-agent: GPTBot Disallow: /lecture/ User-agent: LinkedInBot Allow: /account/accomplishments/ User-agent: Twitterbot Allow: /account/accomplishments/ User-agent: meta-externalagent Disallow: /