Free tools

Robots.txt Examples

Robots.txt file content for everydayhealth.com.

Robot.txt file for: everydayhealth.com

      User-agent: *
Disallow: /mail/
Disallow: /flash/
Disallow: */destroy/
Disallow: /symptom-checker/SymptomChecker/GetSymptoms 
Disallow: /ACCOUNT/LogOn/1  
Disallow: /solutions/managing-child-allergies/food-allergy-recipe-swap/
Disallow: /testpage/
Disallow: /v2/
Disallow: /sitefinity/
Disallow: /socialshare/
Disallow: /drugs/search
Disallow: /search/
Disallow: /v2/esi/getbibbliowidget/v2/
Disallow: /api/
Disallow: /esi/
Disallow: /drugs-supplements/

User-agent: GPTBot
Disallow: /

User-agent: PerplexityBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Claude-Web
Disallow: /

Sitemap: https://www.everydayhealth.com/sitemap.xml