Free tools
Robots.txt Examples
Robots.txt file content for oup.com.
Robot.txt file for: oup.com
User-agent: * Disallow: /web_lifecycle/* SITEMAP: https://global.oup.com/academic/sitemap.xml SITEMAP: https://elt.oup.com/sitemap.xml SITEMAP: https://global.oup.com/education/sitemap.xml SITEMAP: https://global.oup.com/ukhe/sitemap.xml SITEMAP: https://global.oup.com/ushe/sitemap.xml Disallow: /academic/store/ Disallow: /academic/ecommerce/ Disallow: /academic/promo/ Disallow: /academic/promojump/ Disallow: /academic/-/l/cart* Disallow: /academic/promotions/ Disallow: /academic/search Disallow: /academic/advanced_search Disallow: /emetest/ Disallow: /academic/xyz/ Disallow: /education/store/ Disallow: /education/searchresults Disallow: /education/advancedsearch Disallow: /education/basket/ Disallow: /education/account/ Disallow: /ukhe/xyz/ Disallow: /ushe/xyz/