Konigle

November 26, 2024

Robots.txt for etymonline.com

        User-agent: Googlebot
Disallow: /admin/
Disallow: /app/feedback

User-agent: Bingbot
Disallow: /classic
Disallow: /classic/
Disallow: /admin/
Disallow: /app/feedback

User-agent: CCBot
Disallow: /

User-agent: ChatGPT-User
Disallow: /

User-agent: SemrushBot
Disallow: /

User-agent: SemrushBot-BA
Disallow: /

User-agent: SemrushBot-SA
Disallow: /

User-agent: SiteAuditBot
Disallow: /

User-agent: SplitSignalBot
Disallow: /

User-agent: SemrushBot-COUB
Disallow: /

User-agent: SemrushBot-SI
Disallow: /

User-agent: SemrushBot-SWA
Disallow: /

User-agent: SemrushBot-CT
Disallow: /

User-agent: SemrushBot-BM
Disallow: /

User-agent: SemrushBot-SEOAB
Disallow: /

User-agent: Teleport
Disallow: /

User-agent: TeleportPro
Disallow: /

User-agent: dotbot
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: AhrefsBot 
Disallow: /

User-agent: *
Allow: /search
Allow: /word/
Disallow: /web/login
Disallow: /classic
Disallow: /classic/
Disallow: /admin/
Disallow: /app/feedback

Sitemap: https://www.etymonline.com/landings.xml
Sitemap: https://www.etymonline.com/sitemap.xml
Sitemap: https://www.etymonline.com/sitemap-article.xml

Sitemap: https://www.etymonline.com/locale-sitemap/home.xml
Sitemap: https://www.etymonline.com/sitemaps/words-locale.xml

Keep Reading