Free tools
Robots.txt Examples
Robots.txt file content for dictionary.com.
Robot.txt file for: dictionary.com
# Production robots.txt User-agent: GPTBot Disallow: / User-agent: deepcrawl Disallow: / User-agent: seerbot/1.0 Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: * Disallow: /5480.iac. Disallow: /go/ Disallow: /audio.html/ Disallow: /houseads/ Disallow: /askhome/ Disallow: /23219321/iac. Disallow: /*,* Disallow: /browse-2/ Allow: / Sitemap: https://www.dictionary.com/sitemap-dict-index.xml Sitemap: https://www.dictionary.com/e/sitemap_index.xml