November 26, 2024

Robots.txt for dictionary.com

        # Production robots.txt

User-agent: GPTBot
Disallow: /

User-agent: deepcrawl
Disallow: /

User-agent: seerbot/1.0
Disallow: /

User-agent: Applebot-Extended
Disallow: /

User-agent: *
Disallow: /5480.iac.
Disallow: /go/
Disallow: /audio.html/
Disallow: /houseads/
Disallow: /askhome/
Disallow: /23219321/iac.
Disallow: /*,*
Disallow: /browse-2/

Allow: /
Sitemap: https://www.dictionary.com/sitemap-dict-index.xml
Sitemap: https://www.dictionary.com/e/sitemap_index.xml