Free tools

Robots.txt Examples

Robots.txt file content for msdmanuals.com.

Robot.txt file for: msdmanuals.com

      User-agent: *
Crawl-delay: 5
Disallow: */sitecore
Disallow: */Sitecore
Disallow: */custom/
Disallow: */Custom/
Disallow: */news/external/
Disallow: */News/External/
Disallow: */pagerevalidation
Disallow: */multimedia/zk/
Disallow: */Multimedia/zk/
Disallow: */multimedia/Zk/
Disallow: */Multimedia/Zk/
Disallow: */multimedia/ZK/
Disallow: */Multimedia/ZK/
Disallow: */professional/drug-names-generic-and-brand
Disallow: */home/drug-names-generic-and-brand
Disallow: */home/monograph/*
Disallow: */professional/monograph/*
User-agent: Googlebot-Image
Disallow: *blausen*
#Baiduspider
User-agent: Baiduspider
Disallow: /zh/

Sitemap: https://www.msdmanuals.com/de/sitemap.ashx
Sitemap: https://www.msdmanuals.com/sitemap.ashx
Sitemap: https://www.msdmanuals.com/es/sitemap.ashx
Sitemap: https://www.msdmanuals.com/fr/sitemap.ashx
Sitemap: https://www.msdmanuals.com/it/sitemap.ashx
Sitemap: https://www.msdmanuals.com/ja-jp/sitemap.ashx
Sitemap: https://www.msdmanuals.com/ko/sitemap.ashx
Sitemap: https://www.msdmanuals.com/pt/sitemap.ashx
Sitemap: https://www.msdmanuals.com/ru/sitemap.ashx
Sitemap: https://www.msdmanuals.com/ru-ru/sitemap.ashx
Sitemap: https://www.msdmanuals.com/ar/sitemap.ashx
Sitemap: https://www.msdmanuals.com/vi/sitemap.ashx
Sitemap: https://www.msdmanuals.com/uk/sitemap.ashx
Sitemap: https://www.msdmanuals.com/hi/sitemap.ashx
Sitemap: https://www.msdmanuals.com/sw/sitemap.ashx
User-agent: Googlebot-Image
Disallow: *blausen*