Free tools

Robots.txt Examples

Robots.txt file content for clevelandclinic.org.

Robot.txt file for: clevelandclinic.org

      User-agent: *
Disallow: /Data
Disallow: /Utility
Disallow: /svc
Disallow: /service-pages
Disallow: /Search
Disallow: /search
Disallow: /*/Search
Disallow: /*/search
Disallow: /ccf/media/files/*Guide.pdf
Disallow: /ccf/media/files/*guide.pdf
Disallow: /*?date=*
Disallow: /*?q=*
Disallow: /*?view=*
Disallow: /services/heart/mychart-welcome
Disallow: /services/heart/mychart-welcome/
Disallow: /Cache/ClearCache/
Disallow: /AtoZ/HealthInformationLetterStatus/
Disallow: /AtoZ/HealthInformationPages/
Disallow: /AtoZ/HealthInformationRibbonLinkVisibility/
Disallow: /AtoZ/InstitutesDepartmentsLetterStatus/
Disallow: /AtoZ/InstitutesDepartmentsPages/
Disallow: /Ribbon/LocationSpecialtyLists/
Disallow: /Provider/PhysicianRatings/
Disallow: /Provider/Image/
Disallow: /Video/Panel/
Disallow: /ClinicalTrial/JsonResults/
Disallow: /PatientStory/JsonResults/
Disallow: /PatientStory/ForSearchIndex/
Disallow: /Location/WaitTimes/
Disallow: /treatment-guides/
Disallow: /hello-world
Disallow: /test-health
Disallow: /hello-world/
Disallow: /test-health/
Sitemap: https://my.clevelandclinic.org/sitemap.xml